System device and method for configuring and operating interoperable device having player and engine

ABSTRACT

System, device, method, and computer program and computer program products for providing communicating between devices having similar or dissimilar characteristics and facilitating seamless interoperability between them. Computer program software and methods of and systems and devices for sharing of content, applications, resources and control across similar and dissimilar permanently or intermittently connected electronic devices. Devices, systems, appliances, and the like communicating and/or interoperating within the framework provided. Interoperability device, such as a DartDevice, provides a highly interoperable device by virtue of its running a compliant player such as a DartPlayer containing an engine such as a DartEngine and at least one communications protocol for connecting to other devices.

RELATED APPLICATIONS

This application claims the benefit of priority under 35 U.S.C. 119(e)to U.S. Provisional Patent Application Ser. No. 60/577,971 filed 8 Jun.2004 entitled Architecture, Apparatus And Methods Thereof For AnEfficient, Low Cost, Seamless Device Interoperability Software Platformand naming inventor Daniel Illowsky; which application is herebyincorporated by reference here in its entirety.

The following co-pending U.S. Utility Patent Applications and PCTInternational Patent Applications are also related applications and eachis incorporated herein by reference in its entirety:

-   -   Attorney Docket No. 186245/US/8 filed 8 Jun. 2005 and entitled        Method And System For Device Recruitment Interoperability And        Assembling Unified Interoperating Device Constellation;    -   Attorney Docket No. 186245/US/9 filed 8 Jun. 2005 and entitled        Method System and Data Structure For Content Renditioning        Adaptation And Interoperability Segmentation Model;    -   Attorney Docket No. 186245/US/5 filed 8 Jun. 2005 and entitled        Method and System for Specifying Device Interoperability Source        Specifying Renditions Data and Code for Interoperable Device        Team;    -   Attorney Docket No. 186245/US/2 filed 8 Jun. 2005 and entitled        Device Interoperability Framework and Method For Building        Interoperability Applications For Interoperable Team of Devices;    -   Attorney Docket No. 186245/US/14 filed 8 Jun. 2005 and entitled        Device Interoperability Tool Set and Method For Processing        Interoperability Application Specifications into Interoperable        Application Packages;    -   Attorney Docket No. 186245/US/15 filed 8 Jun. 2005 and entitled        Device Interoperability Format Rule Set and Method for        Assembling Interoperability Application Package;    -   Attorney Docket No. 186245/US/16 filed 8 Jun. 2005 and entitled        Device Interoperability Runtime Establishing Event Serialization        and Synchronization Amongst a Plurality of Separate Processing        Units and Method for Coordinating Control Data and Operations;    -   Attorney Docket No. 186245/US/11 filed 8 Jun. 2005 and entitled        Method and System For Linear Tasking Among a Plurality of        Processing Units;    -   Attorney Docket No. 186245/US/10 filed 8 Jun. 2005 and entitled        Method and System For Vertical Layering Between Levels in A        Processing Unit Facilitating Direct Event-Structures And        Event-Queues Level-to-Level Communication Without Translation;    -   Attorney Docket No. 186245/US/7 filed 8 Jun. 2005 and entitled        System And Method For Application Driven Power Management Among        Intermittently Coupled Interoperable Electronic Devices;    -   Attorney Docket No. 186245/US/17 filed 8 Jun. 2005 and entitled        System And Method For Interoperability Application Driven Error        Management and Recovery Among Intermittently Coupled        Interoperable Electronic Devices;    -   Attorney Docket No. 186245/US/4 filed 8 Jun. 2005 and entitled        Device and Method For Interoperability Instruction Set;    -   Attorney Docket No. 1862451US/3 filed 8 Jun. 2005 and entitled        Method and System For Customized Programmatic Dynamic Creation        of Interoperability Content;    -   Attorney Docket No. 186245/US/12 filed 8 Jun. 2005 and entitled        Method and System for Interoperable Content Player Device        Engine;    -   Attorney Docket No. 186245/US/18 filed 8 Jun. 2005 and entitled        Method and System for Interoperable Device Enabling Hardware        Abstraction Layer Modification and Engine Porting;    -   Attorney Docket No. 186245/US/13 filed 8 Jun. 2005 and entitled        System Method and Model For Maintaining Device Integrity And        Security Among Intermittently Connected Interoperating Devices;    -   Attorney Docket No. 186245/US/19 filed 8 Jun. 2005 and entitled        System Method and Model For Social Synchronization        Interoperability Among Intermittently Connected Interoperating        Devices;    -   Attorney Docket No. 186245/US/20 filed 8 Jun. 2005 and entitled        System Method and Model For Social Security Interoperability        Among Intermittently Connected Interoperating Devices;    -   Attorney Docket No. 186245/US/6 filed 8 Jun. 2005 and entitled        System Device and Method for Configuring and Operating        Interoperable Device having Player and Engine;    -   Attorney Docket No. 186245/US/21 filed 8 Jun. 2005 and entitled        Method and System For Specifying Generating and Forming        Intelligent Teams of Interoperable Devices;    -   Attorney Docket No. 186245/US/22 filed 8 Jun. 2005 and entitled        Method and System For Configuring and Using Virtual Pointers to        Access One or More Independent Address Spaces;    -   Attorney Docket No. 186245/PCT/2 filed 8 Jun. 2005 and entitled        Architecture Apparatus And Method For Seamless Universal Device        Interoperability Platform; and    -   Attorney Docket No. 186245/PCT/3 filed 8 Jun. 2005 and entitled        Architecture Apparatus And Method For Device Team Recruitment        and Content Renditioning for Universal Device Interoperability        Platform.

FIELD OF INVENTION

The present invention generally relates to systems, devices, methods,and computer program software products for providing communicatingbetween devices having similar or dissimilar characteristics andfacilitating seamless interoperability between the devices; and moreparticularly, to software and methods of and systems and devices forsharing of content, applications, resources and control across similarand dissimilar permanently or intermittently connected electronicdevices.

BACKGROUND

In an era when there has been a vast expansion in the number and type ofelectronic devices, particularly portable and wireless devices, as wellas an expansion in the types of application programs and device types,there has been a corresponding need for data and code to be shareddirectly between these diverse heterogeneous different device types inorder to carry out the intent of applications which can only beaccomplished by employing the electronic and programmatic resources of aplurality of devices. A need has also arisen and continues to grow for adevice user to be able to communicate with other devices that may or maynot be set up in advance for the type of communication or data transferor sharing that the user desires. For example, a user may have or wishto create a picture collection on a digital camera and then to be ableto transfer that collection of pictures directly to a personal dataassistant (PDA) type device, television, or projector for viewing, or toa storage device for storage or to a printer. The user may also oralternatively wish to transfer code which implements a sequenced slideshow encapsulating the pictures, titles, index of slides, and the liketo another device, such as to a device that has a larger screenresolution or better graphics capability than the device on which theslide images reside. The user may also want to be able to select andprint some subset of pictures in the slide show on an available printer.There are many other examples of such code, data and content sharing.

Conventional Interoperability Issues, Problems, and Limitations

The sharing of data and code along with the sharing of associated devicecomputing resources, and device control between similar (homogeneous)and dissimilar (heterogeneous) devices or device types is known in theart as “device interoperability,” or simply as “interoperability”. Someof the necessary and optional enhancement issues involved in providingthis interoperability include the issues of: (i) content adaptation;(ii) content format; (iii) device drivers; (iv) device to devicecommunication; (v) individual device resources and capabilities; (vi)application programs resident on the devices; (vii) loading applicationprograms on the devices; (viii) costs associated with providing deviceresources to support interoperability; (ix) power or energy managementof interoperable devices; and (x) robustness of code executing in aninteroperability environment where connections between devices may beintermittent and unreliable. Furthermore, (xi) the scope of thedevelopment, deployment and testing efforts necessary to enableinteroperability; (xii) the reliability problems inherent in havingindependently developed and or distributed interoperability componentseven where detailed interoperability standards exist; and (xiii) thedifficulty of end users having to have a high level of technicalknowledge and spend appreciable amounts of time and effortsadministering interoperability; (xiv) the security of interoperabilitydevices, data and content; (xv) the size, performance, power managementand cost tradeoffs that can be made with respect to interoperabilityinfrastructure, raises additional issues. These issues are addressed inadditional detail below.

With respect to content adaptation, there is a need for intelligentscaling or adaptation of the content in terms of such parameters(application and data type dependent) as picture size, user interface,controls and special effects, content format, features, and the likethat needs to be taken care of when transferring data, applicationprograms (applications), or control from one device type to another.These are collectively referred to as “adaptation.” The better thesophistication of the adaptation when sharing content, applications, andcontrol, the larger the set of interoperable devices; and the moreadvanced the features on each device, the more efficient the transfer ofdata, information, and/or other capabilities can be, and the easier thedevices and code data and content are to use to carry out applications.

A second interoperability issue arises from the undesirable requirementthat the user may generally need to specify or at least consider thecontent format. If the user desiring interoperability with anotherdevice is not familiar with the content format and/or how the otherdevices will deal with the content format even if it can be communicatedto the other device, this factor alone may preclude interoperability.

A third interoperability issue arises from the undesirable requirementthat the user may generally need to specify, consider, or carry out theloading of one or more special purpose drivers, code, data or content onone or more devices before interoperability can be carried out.

A fourth interoperability issue arises from the undesirable requirementthat the user specify, consider, or select the physical communicationsmechanisms and protocols to be employed in a communication between theuser's device and one or more other devices, each of which may have orrequire a communication mechanism, protocol, interface, or the like.

A fifth interoperability issue arises from the undesirable requirementthat the user may need to consider or chose which devices will have thecapabilities and memory, processor and other features necessary tointeroperate with his or her device or with the data or applicationsrequired.

A sixth interoperability issue arises from the undesirable requirementthat the user may need to specify, consider, and/or load theapplications that must reside on some or all of the involved andpotentially interoperable devices.

A seventh interoperability issue arises from complete or partialapplication failure due to missing, outdated, or incompatible version ofcode, data or content on one or more devices.

An eighth interoperability issue arises from the undesirable requirementthat devices need to have all the code to carry out applications thatwill be needed resident at the time of manufacture or at some time priorto the need for them arising, or be explicitly loaded on some or all ofthe devices by the user.

A ninth interoperability issue arises from the monetary cost associatedwith providing the amount of processor or CPU resource, memoryresources, electronic gates or logic, or other physical infrastructurenecessary to implement the communications and other protocols andapplications on devices intended to interoperate.

A tenth interoperability issue arises from the desirability of providingeffective power or energy management methodologies to extend batterylife or reduce the size of the batteries needed for portable or mobiledevices that are intended to interoperate. Although not specificallyrequired for short term interoperability, such power management ishighly desirable so that interoperating with other devices will notcreate such a battery power drain on such devices that users wouldrarely use the capabilities or be hesitant to permit another user toaccess their device.

An eleventh interoperability issue arises from the need for a degree ofrobustness of applications which need to continue to operate in anenvironment where connections between devices are often intermittent ortransient and unreliable. For example, code to carry out an applicationon a first device that is interoperating with and in communication witha second device should not itself freeze, hang, or otherwise cause amajor problem or result in the device itself freezing, hanging, orcausing a major problem when the second device moves out of range orotherwise fails to reply to a communication from the first device.Furthermore, it is desirable for all the code, data and contentnecessary to carry out an interoperability application to beautomatically restored and updated if such a second device becomesreliably available again.

A twelfth interoperability issue arises from the unreliability ofapplications where devices are produced by independent manufacturers,based on interoperability standards which are inherently weak in theirability to predict realistic and future device needs and capabilities,and in the ability of programmers or circuit designers to completely andcorrectly understand, implement and have such implementations correctlydeployed. A thirteenth interoperability issue arises from the slow speedof executing code which cannot rely on optimizations necessary forgraphics, video, sound, etc.

A fourteenth interoperability issues arises from the lack ofavailability of interoperable code, data and content to carry outapplications and devices which might be employed for interoperabilitydue to all the issues listed above which discourages both users andproviders.

These fourteen interoperability issues are merely exemplary of the typesof issues that do or may arise and are not intended to be a completelist or to identify issues that arise in all situations. For example,interoperability between two identical devices that are intended tointeroperate with each other at the time of manufacture may not presentany or all of the issues described here, but this type of homogeneousdevice interoperability does not represent the more common situationthat device users are faced with today, and modest attempts to addressheterogeneous device interoperability issues have been incomplete, notvery insightful, and clearly not successful.

Conventional Static and Procedural Solution Attempts

Conventional attempts at providing interoperability solutions havegenerally fallen into two categories, namely (i) static interoperabilitysolutions (“static”), or (ii) procedural interoperability solutions(“procedural”). Conventional static solutions require each device tosupport the same specific communications protocols, and send specificrigidly specified data structures with fixed field layout. In staticapproaches, the semantics, code, and display capabilities must beexistent on all devices before interoperability can be establishedbetween those devices. Each content type, application, or devicecapability must be known, implemented and installed at the time ofmanufacture of all devices involved; or alternately, the user mustinstall application programs, protocols, and/or drivers as requiredprior to initiating the desired interoperability of devices, softwaredata or content. As the user may not be a trained information technologyprofessional, or may not know or have a copy of the driver, application,operating system component, protocol, or the like, it may be impossibleto provide the desired interoperability within the time available.Furthermore, often it is necessary with static solutions to implement aspecific set of static solutions.

For example, the sharing of a set of pictures with slideshowcapabilities between a digital camera and a television or display device(TV) might require a common static protocol, such as for example, aBluetooth wireless capability for sending the slide image data and slideorder or sequence information to a TV. It would also require a staticcontent format for the slides and slide order information to berecognized on the TV as something it knows how to deal with. And atleast one static slide show program that can render and control a slideshow with the specific content format must exist on both the TV and thedigital camera. The user may or may not have to separately initiatetransfer of the images or pictures and slide order information, find andassociate the information on the TV, and run the correct slide showapplication on the TV. Depending on the sophistication of the staticslideshow programs on both sides, the controls on the digital camera mayor may not be useable to control the slide show on the TV which wasinitiated on the camera. Where such camera based control is not possiblesome other mechanism for control may necessarily be provided. Staticapproaches can result is highly optimized solutions to well understoodspecific applications known at the time of manufacture of all the devicetypes which can interoperate. Static approaches however have majorlimitations, including the requirement for most all capability to beknown and custom implemented at the time of manufacture, limited abilityto upgrade or fix errors after manufacture; and a conventionalrequirement that each static program implementation must be correctlyand completely ported to run on the different devices and exist on alldevices prior to interoperation. Often this is accomplished by theloading and updating of specific drivers for specific applications,communications mediums and the desired set of devices whereinteroperability is required.

Even when static solutions are available, reliability is compromised dueto the inevitability of different versions of standards andapplications. Hence when two devices wish to share data or procedures,failure can occur when the devices adhere to different versions of thestandard, or the programs adhere to different versions of the standard,or different versions of the applications reside on the devices.Additional reliability problems arise from inadvertent errors orshortcuts made in the independent implementations of the set ofstandards used to interoperate. Such standards implementations mayinteract in unpredictable ways when any two implementations attempt towork together. In general it is often impractical or impossible to testall the permutations of all the standard implementations across all setsof devices, especially as all target devices which with an initiatingdevice must interoperate with did not exist at the time of manufacturingof the initiating device.

One of the more significant limitations to static approaches is that theamount of work to make a number of N devices or applicationsinteroperable grows very quickly as N for the number of devices and/orapplications gets larger. Manufacturers currently are flailing atcreating hundreds of static standards for content types, applicationprograms (“programs”), communications protocols, and the like to try tomake even limited size sets of devices interoperable over an everincreasing large set of devices and applications. This alsoconventionally requires that every device have the memory, screen size,controls and processor and battery power to support every staticsolution for every desired interoperability option across all desiredinteroperable devices. Otherwise true device and applicationinteroperability is not achieved. To better illustrate this conventionalproblem and limitation, consider that currently, adaptation requires asoftware engineering development project for each type of device (Ndevices) that has to share with each other type of device (N−1 devices).From a development point of view this is an N×N or N² order problembecause in a universe of N device types that all wish to interoperatewith each other, there are N×(N−1) adaptations to consider, develop,implement and test.

Moreover, as the number of devices increases and the requiredadaptations rise toward N², the expense and difficulty of attaining ahigh-quality product tends to increase at an even faster rate due to theincreased overall complexity. This is because the difficulty ofattaining high-reliability and quality software and hardware solutionsincreases as overall complexity increases. This isn't purely a “size ofsource code” issue, but is due to just the kind of factors prevalentwhen trying to get devices from different manufacturers to worktogether, including unpredictability of behavior, unpredictability ofevents, unknown future capabilities, and so on.

For example, as the number of devices increases from 5 to 6,interoperability adaptation requirements increase according to therelationship N×(N−1) from 20 to 30. And as this increases the overallcomplexity of the project has grown even faster.

This N-squared order problem wherein getting N devices to work togetherrequires substantially N² adaptations is illustrated in FIG. 1. It willbe appreciated that conventional inter-device cooperation using a staticapproach may be relatively simple for a user to use but requires a highdegree of development, administration and deployment effort andcontinual updating to maintain compatibility and interoperabilitybetween old devices, applications, and data types, and new devices,applications, and data types.

With reference to FIG. 2, there is shown the interactions forinter-device cooperation or limited interoperability for just eightdevice types using a brute force approach requiring fifty-sixadaptations and additional fifty-six test procedures.

Separate from the N² problem of the brute force method, most staticapproaches also involve combining a number of standard or standardsefforts. The Microsoft originated UPnP (Universal Plug and Play)approach has perhaps the largest following and scope of the staticapproaches. UPnP is a static non-procedural approach to solving some ofthe problems associated with device interoperability by incorporating aset of static (non-procedural based) standards, and attempting toenumerate all the different classes of devices and services, each with adifferent XML or data structure based description. However, even theUPnP approach suffers significant limitations, some of which are brieflydescribed below.

First, UPnP is heavyweight in that it requires large collections ofmodules and code, power, and memory to run. This makes it unsuitable forthin low cost battery powered devices that may have a very modestprocessor, little random access memory, a small battery capacity.

Second, UPnP offers little content or feature optimization capability.UPnP generally assumes one size application, content, or user interfacewill work well on all involved devices. This may have been a reasonableassumption a decade ago for Microsoft Windows based desktop personalcomputers (PCs), but is now a poor assumption and basis for operation ina world filled with devices that must interoperate that are as differentas a pager, a digital camera, and a personal computer, not to mentionthe likely set of hybrid and diverse electronic devices to arise in thenext few decades.

Third, UPnP offers only a limited set of user interfaces that do notmeet the needs of the large set of diverse devices now available.

Fourth, UPnP requires programs and drivers that are needed to performthe requested task to reside on all devices before they can be used.

Fifth, while the intent of UPnP is at least to partially avoid theN-Squared (N²) problem, the reality is that using UPnP as a basis forthe interoperability would still require a massive N-Squared (N²)development/deployment/testing effort, as described above, to bring outnew applications which require programs, code, data and content to beported, distributed and tested for all interoperating devices if the allthe permutations of independent implementations of complex standards isto result in reliable interoperability.

Sixth, UPnP programs, devices, content and standards must all besynchronized so that the same or at least compatible versions andupdates are deployed simultaneously

Seventh, as device and content capabilities evolve, existing UPnPprograms, data and content based devices tend to fail to support the newdevice.

Eighth, the costs of maintaining compatibility of existing device,standards including UPnP, and applications versions increases theoverall project complexity ever more rapidly.

Ninth, as the project complexity increases, due to the problems inherentin the complexity and diversity of UPnP as a standards-static-basedapproach, ease-of-use and reliability degrade.

Tenth, UPnP would still not address the requirements imposed by thefrequent need to have one or many data structures sent between devices,especially where this data or these data structures are expressed in ahuman readable text format that require considerably more transmissionbandwidth and time than binary representations. Furthermore, many of thedata structures to be sent between devices are expressed in XML, a humanreadable text format, rather than in a binary or less generalizedformat, where using XML format requires significantly more CPUoperations, memory, and/or software program code size to perform the CPUintensive parsing operations required for XML.

Finally relative to a few of the limitations imposed by conventionalstatic approaches to device and application interoperability, staticstandards often limit the number of protocols, content types andapplication types to reduce the overall complexity and size of standardsbased implementations. An example is that UPnP only allows TCP/IP as abase communication protocol. This effectively eliminates the efficientuse of other important existent communications protocols such asBluetooth, USB, MOST, and IR and all other non-TCP/IP protocols.

Conventional Procedural Solution Attempts

An alternative to the static standards approach relies on creating aprocedural standard. Procedural standards techniques implemented inhardware or emulated in software are ubiquitous. There exist a largenumber of hardware microprocessors, each with an instruction set andinterfaces optimized to different classes of problems, and there arenumerous higher level software emulated instruction sets andenvironments existent, that are optimized around specific task sets.These include for example, Java (an approach generally optimized forportability and ease of programming), PostScript (an approach generallyoptimized to represent printed pages and printer control functions), andStorymail Stories (generally optimized for efficiently representing avery broad range of rich multimedia messages). Java and PostScript arewell known in the computer arts. Aspects of Storymail Stories andrelated systems and methods are described, for example, in U.S. PatentApplication Publication No. 20030009694 A1 published 9 Jan. 2003entitled Hardware Architecture, Operating System And Network TransportNeutral System, Method And Computer Program Product For SecureCommunications And Messaging; and naming Michael L Wenocur, Robert W.Baldwin, and Daniel H. Illowsky as inventors; in U.S. Patent ApplicationPublication No. 20020165912 A1 published 7 Nov. 2002 and entitled SecureCertificate And System And Method For Issuing And Using Same, and namingMichael L Wenocur, Robert W. Baldwin, and Daniel H. Illowsky asinventors; and in other patent applications.

Procedural interoperability approaches, typically involve establishingor otherwise having or providing a common runtime environment on alldevices that are to interoperate, so that programs, procedures, data andcontent can be sent between devices in addition to static datastructures and static applications. Currently one leadinginteroperability procedural solution is the Java platform along with theJINI extensions to it. As an example of a Java-based proceduralapproach, a slideshow written in Java could encapsulate or reference thepictures and slide ordering or sequence data, interrogate the otherdevice, adapt the content to the other device, and send the informationand a Java slideshow program to the TV. The Java slideshow program couldbe run on the camera after manufacture and enable interoperability witha Java enabled TV even if the slide show program did not pre-exist onthe TV.

While Java has been widely deployed and has some limited success inproviding correspondingly limited interoperability, it has seriousdeficiencies that have prevented its broad use, especially for smallmobile devices where cost, power efficiency, processor efficiency,memory efficiency for storing program code, data, and temporary buffersare very important issues. Also, the Java Virtual Machine (VM) approachto binary compatibility of applications running on different devices isin conflict with the very reason the devices exist. Java and otherconventional procedural interoperability approaches have severlimitations. Five exemplary limitations are described below.

First, the Java Virtual Machine approach makes or at least attempts tomake all devices look like the exact same virtual computer toapplications in order to allow the same binary code (Java binary code)to run on all devices. In order to maintain binary compatibility, it isnecessary to avoid attempts to access device capabilities that were notpredefined as part of the Virtual Machine definition and implementation.Thus binary compatibility is lost across multiple devices if nativefunctions are needed to access capabilities of any device that are notpart of the common Virtual Machine definition. Since most non-PC devicehardware and software are most often specialized or optimized for aparticular purpose, form factor, price point, user interface orfunctionality, it is often the case that their basic unique nativefunctions or capabilities must be accessed for the applications mostoften targeted for the device. For most portable or special purposedevices, the very reason for their existence is because there is a needfor uniquely different capabilities and functions. This runs counter tothe Java Virtual Machine approach of hiding the differences betweendevices to make them all look the same to the application.

Secondly, Java is a general purpose language optimized for ease ofprogramming at the expense of efficient execution and efficient memoryuse. Therefore, it will not be the most efficient or effective solutionfor many thin devices with modest processing capability and littleavailable memory or where cost is important.

Thirdly, multimedia content response times cannot be assured using aJava procedural approach. Most Java programs are heavily reliant on thefrequent allocation and de-allocation of varying size memory structurescausing memory fragmentation. This memory fragmentation often leads toperiods when the processor within the device must stop rendering thecontent while it performs garbage collection within the memory. Userswill often experience a breakup in smoothness of audio and videorendering when this occurs.

Fourthly, Java presents significant speed and size issues. Java and itsassociated technologies and libraries necessary for interoperability arerelatively heavyweight and require a relatively large number of CPUcycles for execution and relatively large amounts of memory for storage.Interoperability programs written in Java that include user interfaces,multimedia rendering, device enumeration, robust cross platform deviceinteroperability, dynamic adaptation of code and data to send todifferent types require a large amount of code to be written andexchanged because all of these functions must be built up usinglibraries, or special purpose Java code sequences, as none of theseoperations are native to the Java instruction set or environment. Theresult is that Java programs for interoperability are large and slow,limiting their use on devices where limited processor power, batterylife or cost are important issues. Where the devices do not havesufficient resources a Java based interoperability solution is notpossible.

Fifthly, Java at beast provides a limited and incomplete baseimplementation for interoperability. This makes it necessary for a largenumber of libraries to be existent on all devices to interoperate, orhave a high speed always-on connection to servers which contain thenecessary program code. Performance for operations not included in thebase instruction set of Java must be provided in the Java languageitself, greatly limiting the runtime performance verses that of nativecode that might otherwise be used to implement these operations. Missinginteroperability base operations include native support for: (i)multimedia animation playback; (ii) adaptation of programs, data,content, user interface or controls to target other devices; (iii)computer generation of custom programs so that devices that originatecontent can automatically and easily marry that content withinteroperability programs; (iv) device, service and resource discoveryover a wide variety of protocols; (v) synchronization and/orserialization of processes running in different devices; (vi) devicepower management; (vii) application and synchronization recovery whendevices intermittently lose and regain their connections.

Often when a Java VM specification proves to be deficient for a class ofdevices or applications, a new Java VM specification arises to addressthe now known native support needs of this new class of devices;however, Java programs written for one VM are not generally binarycompatible or interoperable with devices which conform to different VMspecifications. Java VM specifications exist for various devicesclasses, including the J2ME, MIDP 1.0, MIDP 2.0, and CDC, but thisproliferation of ever more non-interoperable Java VM specifications andimplementations continues to cause a form of fragmentation of the typesand forms of programs and devices that achieve even a small degree ofinteroperability through the use of Java VMs.

Xerox Palo Alto Research Complex (PARC) has announced a variation on theJava plus Jini interoperability technologies which they call “Obje”.Obje is explicitly based on Java, or as an alternative, an unspecifiedand unrealized similar virtual machine based technology. While Objepoints to some ways of providing procedural methodologies needed toeffectively team devices and eliminate the requirements for all devicesto have the programs ported or resident on all machines, it is expectedthat Obje implementations will have similar capabilities and limitationsas the Java plus Jini approach as they offer no details to indicate anydivergence from the Java VM model for the procedural base to be used

PostScript, another procedural approach, has been around for aconsiderable time and provides a printed page description language whichhas been very effective at establishing a high degree ofinteroperability between PostScript documents and PostScript printers.PostScript documents are programs which when executed on a PostScriptsoftware engine inside a printer, control the hardware printer engineand recreate the image of printed pages while taking advantage of thehighest resolution possible on the printer that it finds itself on.PostScript is largely limited to the interoperability of documents andprinters. Some of the reasons for this limitation include the fact thatPostScript documents are expressed in human readable text. This expandsthe size of documents and programs greatly over binary programs. Thetext requires parsing operation when the programs are run, requiringmore processor cycles and scratch memory then would be necessary if theprograms were expressed in binary. Furthermore, PostScript does notprovide any significant native support for: (i) Multimediavideo/audio/animation playback; (ii) adaptation of application, data,content, user interface or controls to target other devices; (iii)device, service and resource discovery; (iv) synchronization andserialization of programs running in multiple devices; (v) device powermanagement; (vi) finding and using other devices; (vii) maintainingrobust connections between devices; or (viii) efficient access tovarious storage mediums, including the common flash memory now common ondevices.

Storymail Stories provide a variable length procedural instruction setdesigned for encapsulating multimedia messages. Aspects of StorymailStories and related systems and methods are described, for example, inU.S. Patent Application Publication No. 20030009694 A1 published 9 Jan.2003 entitled Hardware Architecture, Operating System And NetworkTransport Neutral System, Method And Computer Program Product For SecureCommunications And Messaging; and naming Michael L Wenocur, Robert W.Baldwin, and Daniel H. Illowsky as inventors; in U.S. Patent ApplicationPublication No. 20020165912 A1 published 7 Nov. 2002 and entitled SecureCertificate And System And Method For Issuing And Using Same, and namingMichael L Wenocur, Robert W. Baldwin, and Daniel H. Illowsky asinventors; and in other patent applications.

The Storymail invention including the Storymail Story structure andassociated technologies were invented by Daniel Illowsky the sameinventor as in this patent. This Storymail instruction set allowstrans-coded multimedia content to be represented in a universalprocedural format called, “Stories” which provided significantadvantages over multi-media content representations known theretofore.However, the Storymail technologies did not fully addressdevice-to-device interoperability issues and should one attempt to applythe Storymail technology to achieve device-to-device interoperability,several problems and limitations will quickly become apparent. First,the Storymail instruction set is optimized for small engine sizerequiring the use of a lot of scratch memory. Second, the Storymailinstruction set is burdened with the implementation of a specializedthreading model while running content. Third, Storymail Stories requiretrans-coding of even basic content types, such as JPEG or Bitmap images.Fourth, Storymail Story technology does not provide native support fordevice, service, or resource discovery. Fifth, Storymail Storytechnology has no native base support for synchronization of programsrunning in multiple devices. Therefore, even though Storymail technologyand the universal procedural media format provided significantadvancements over the conventional arts of the day, it does notsatisfactorily solve the device-to-device interoperability issues thatare now apparent in the electronic and computer arts.

It will therefore be apparent that neither static nor current proceduralapproaches provide satisfactory device-to-device interoperabilityparticularly for heterogeneous devices and a priori unknownapplications. The problem is further compounded when the currentclient-server and peer-to-peer inter-device interoperability models areconsidered.

Conventional Client-Server and Peer-to-Peer Models

In the current state of the art, interoperating programs running onmultiple devices generally use either a Client-Server model or aPeer-to-Peer model.

In a client-server environment model, one device (i.e., the server)provides the services and another device (i.e., the client) makes use ofthe services. This allows multiple client devices to take advantage of asingle more capable server device to store and process data, while thelighter weight client device just needs to be powerful enough to makerequests and display the results. A limitation of client-server approachis that generally the server must be registered on a network andaccessible at all times to the clients with a relatively high speedconnection.

In the peer-to-peer environment model, any interoperating device isgenerally assumed to have all the facilities necessary to carry out theapplication. Also in the peer-to-peer model, generally all devices thatinteroperate must have knowledge of the programs or services to beemployed before establishing a connection so that they can agree onappropriate coupling and protocols. Having to have the full capabilitiesto carry out the application, and the need to have the peered programprotocol layers pre-existing on all interoperating devices aresignificant limitations to applying a peer-to-peer model to deviceinteroperability. In practice peer-to-peer devices will often encounterdifferent non-perfect implementations or versions of software which willfail to cooperate do to unpredictable iterations of the differingnon-perfect implementations. To correct these problems, often drivers orother software must be updated, distributed and installed. While thiscan often correct interoperability problems, the administration,implementation, distribution and frustration associated with failuresand the complexity, sophistication and time needed to fix them remains aserious problem of peer-to-peer based device interoperability.

In the world of personal computers the current most popular thoughlimited interoperability platform is the Microsoft Windows™ operatingsystem. Under Microsoft Windows™, theoretically any application binaryimage can run and be useful on any standard PC architecture devicerunning Microsoft Windows™ operating system, regardless of whether thePC was manufactured by IBM, Toshiba, Sharp, Dell or any othermanufacturer. In practical terms, however, even Microsoft Windows haslimitations with interoperability in real-world operating environments.

As computing devices and information appliances diverge from the genericgeneral purpose Personal Computer model into more specific specializeddevices such as mobile phones, mobile music players, remote controls,networkable media players and routers, and a mired of other devices,there is a need for an application platform that allows the quick andefficient development of applications that are not only binarycompatible across all (or at least most) devices, but which can formad-hoc teams of devices on-the-fly over multiple protocols based on theresources of each device. The applications need to be able to spreadtheir execution across all the devices in order to carry outapplications that no one device has all the software, hardware or otherresources needed to implement on its own.

Currently in the state of the art, there is no effective softwareplatform for creating programs that can run and spread themselves acrossmultiple devices, particularly when the devices to be spread to are ofdiverse types and have heterogeneous device hardware, software, andoperating system (if any) characteristics. Although there are manystandardized embedded operating systems, because of the need for theapplications to access the unique capabilities and features of thedevice, including the arrangement of displays and controls, there islittle chance that a program built for one device will be useful if runon another different device, even if both devices use the same embeddedoperating system and processor. Thus, it is clear that there is a greatneed in the art for an improved method and system for providingreliable, easy-to-use device, application program, data and contentinteroperability, while avoiding the shortcomings and drawbacks of theprior art apparatus and methodologies heretofore known.

SUMMARY

The invention comprises a number of inventive systems, devices,apparatus, computer programs and computer program products, proceduresand methodologies that together make device and system interoperabilitysimpler, more reliable, more robust, more powerful, more cost effectiveand more secure than in the heretofore current state of the art. Thetechnology employed is based on procedural interoperability techniques.The invention differs in important and profound ways from existentprocedural interoperability techniques. While the existent techniquesattempt to hide the differences between devices so that the sameexecutable binary image can run on all devices, the invention celebratesand provides access to all the assets of devices to each other so thatthe application can form ad-hoc groups of devices and effectively spreadits execution across the groups of devices much as if all devices in thegroup were one device.

Most existent interoperability technologies are extensions of techniquesdeveloped over the years for powerful general purpose computers incorporate or government networks that are always connected on reliablehigh speed networks, rarely reconfigured, have continuous access toample power and are configured and maintained by trained full-timeprofessionals. These techniques fall short when applied to theinteroperability of mobile, battery powered, intermittently connectedand cost constrained devices now becoming available, and are used byindividuals who are not trained to and would rather not concernthemselves with configuring and maintaining the software and hardwarenecessary for interoperability.

The invention necessarily involves a new software ecosystem ofmethodologies which work together to greatly advance the simplicity,robustness, cost effectiveness, efficiency and security ofinteroperability of mobile and other special purpose devices nowentering the market at an increasing rate.

FIG. 3 shows exemplary components of an embodiment of the inventive newprocedural and software ecosystem collectively referred to as theDartPlatform which itself is a form of interoperability platform. Thesource, here DartSource 100, contains all the code, data and contentneeded to carry out the intent of the application. The DartSource isprocessed by the DartTools 200 into an binary executable applicationpackage which encapsulates all that is needed to carry out the purposeof the interoperability application as originally specified by theDartSource. The binary image of the package conforms to the DartFormat300. The encapsulated applications are called Darts, which are to beexecuted on one or more DartDevices 400. Any device which is running aDartPlayer which is native code executable program which contains theported DartEngine 600, and has at least one communications protocol 401for communicating with other DartDevices is itself a DartDevice capableof running Darts which can extend their execution across to otherDartDevices to make use of their combined capabilities and resources.FIG. 4 shows an Exemplary DartDevice 3000. At the top there are shownthree Dart applications 3001 running on the device 3000. The code ofthese Darts was generated by the DartTools, (See FIG. 3 200), to conformto an inventive DartInstructionSet whose individual operations arecarried out in a secure manner by the portable portion of the DartEngine3010 and the device specific portion of the DartEngine, the HardwareAbstraction Layer (HAL) 3020.

Note that the DartInstructionSet operations carried out by theDartEngine contain processor intensive operations needed forinteroperability such as cryptographic operations, graphics and textprocessing. In addition there is built in support in the engine forinventive interoperability methodologies to be described in more detaillater on in this section. The HAL contains a profile method accessedthrough the PROFILE_INSTRUCTION instructions of the Dart to determinethe devices specific characteristics of the device and itsfunctionality. The HAL also provides methods for the engine to accesscommon Dart standard hardware functions where they exist. Perhaps mostprofoundly inventive is the portion of the HAL which can be used toexpose all the native applications, functionality and resources of adevice to Darts or portions of Darts running on the device which arethen available for use by Darts whose execution extends to otherdevices.

Simplicity of interoperability, reliability and robustness is achievedin part by encapsulating all the code, data and content and the metacode, data and content needed for a particular application purpose tospread itself intelligently and efficiently across heterogeneousdevices. Because all of the application code, data and content runningon the interoperability devices originate from a single Dart packagethere are none of the incompatibility or administration issuesassociated with independently generated and distributed components.Having the data and code packaged together also eliminates commoninteroperability problems which arise from versioning incompatibilitiesbetween the data format and the application program chosen to manage thedata.

There are at least twenty one uniquely described separate inventivesystems, methods, computer programs and computer program products,and/or means which contribute to enhancements of robustness, power,efficiency and security for the interoperability of devices over theexistent interoperability technologies which are largely extensions oftechniques developed over the years for powerful general purposecomputer networks. These innovations are briefly highlighted below andthen described in more detail in subsequent portions of the detaileddescription. There are many more when each useful combination ofseparate inventive system, method, computer program product, and/orother means are combined. Many of the techniques used share thesocialization aspects employed by humans as they form ad-hoc teams andwork together to execute specific tasks. The fast growing world of evermore special purpose and mobile devices which need to form ad-hoc teamsand are only intermittently connected, often has more in common withhuman like collaboration than with the general purpose computer networksfrom which conventional interoperability techniques are borrowed.

Recruitment interoperability model. Recruitment is an advantageousalternative to the existent client/server and peer-to-peer deviceinteroperability models. Recruitment is used by a single softwareapplication package or Dart, to forms teams of devices based on theircapabilities and content and then intelligently spread portions ofitself to the teams of devices which then work together to carry out theintended purpose of the Dart application package.

Renditioning adaptation and interoperability segmentation model.Renditioning allows the segmenting of an interoperability applicationinto a plurality of tightly integrated, yet separately executableprograms. Individual Renditions are chosen during the recruitmentprocess to be sent to run on other devices to provide coordinated accessto the capabilities and content of individual devices.

DartSource/Interoperability Source. DartSource is a method forspecifying all the program renditions and the code content and dataneeded for a packaged Dart interoperability application. DartSourceextends the languages constructs commonly used to specify singleexecutable program targeted to a specific device, into a language whichcan also specify the procedures necessary for intelligent recruitment ofteams of devices and the renditions needed so that there is a suitablerendition to send to run on each recruited device to carry out thatdevice's portion of the intended purpose of the application beingspecified.

DartFramework/Interoperability Framework. DartFramework is the portionof the DartSource provided for use by programmers in buildinginteroperability applications which encapsulate access to many of theadvantageous features of the DartPlatform eliminating the need for theprogrammer to have to understand and implement many of the desiredinteroperability features of the DartPlatform.

DartTools/Interoperability Tools. The DartTools process the DartSourceapplication specification into the Dart application packages.

DartFormat/Interoperability Format. The DartFormat is the rules forputting together a Dart package which encapsulates all the code, data,and content needed for an interoperability applications which can thenbe loaded and run on DartDevices, which contain a running DartPlayer.

DartRuntime/Interoperability Runtime. The DartRuntime is a system forestablishing the tight coordination of control, data and operationsbetween separate processing units of a running Dart whether theprocessing units are running on a single device or across a team ofrecruited devices. This is accomplished by an event driven system whichensures the serialization and synchronization of events flowing thoroughall processing units of the application so that all processing units canhave access to all the directives in the same order needed to coordinateand synchronize the data and operations between the processing units.

Linear Tasking. LinearTasking is an advantageous alternative to theconventional pre-emptive and cooperative threading models commonly usedon most devices so that multiple operations can be specified and run asif their actions were being executed simultaneously. LinearTaskingensures a simple, reliable, flexible and extensibe way for processingunits to coordinate their activities in a very deterministic and easilytested manner. LinearTasking is part of the DartRuntime operating insidea single device.

Vertical Layering. VerticalLayering is an advantageous alternative tothe horizontal layering of protocols in common use on most devices whichrequires different levels of protocols to communicate through allintermediate levels of protocols, often having to translate informationto conform to the differing needs of each protocol interface. Dartprocessing units use VerticalLayering so that regardless of their level,processing units can communicate directly with all other processingunits through the use of event structures and event queues which areaccessible and understandable by all processing units withouttranslation.

Application driven power management. Dart applications built usingLinearTasking and or VerticalLayering as embodied at least partially inthe DartFramework, always keep track of their exact response time needsso that efficient power management techniques such as slowing down theprocessor can extend the lifetime of batteries, limit the amount ofenergy consumed or limit the amount of heat generated on devices. In thecurrent state of the art most applications do not keep track of theirresponse time needs, and if they did would not be able to communicatethese needs through existing layers of protocols which conform tospecifications that do not include interfaces for communicating responsetime needs to the hardware of the device from the application.

Interoperability application driven error recovery. Device to devicewireless communications connections are often unreliable due tointerference, distance limitations, and abrupt shutdowns due to lowbattery power. In conventional horizontally layered protocol softwareimplementations on devices a fatal error in any one layer will result inunrecoverable errors which will be difficult for an application torecover from, both because the application does not have standardinterfaces to easily reestablish the connections and contexts of theconnections, and because conventional application programs do not havemuch infrastructure for tracking and reestablishing shared state betweenapplications running on different devices. The DartFramework keeps trackof shared state, renditioning can be used to easily reestablish loststate between devices and VerticalLayering makes it simple forcommunications errors to be relayed to the Dart and for the Dart torelay recovery information directly to the communications processingunits. Thus Darts running across devices can seamlessly recover fromintermittent complete losses of communications between cooperatingdevices and the recovery of the shared state of the devices when theconnection is restored even where the previously lost device has itselflost all its application state. Interoperability Instruction Set. TheInteroperabilityInstructionSet is used to represent the code portions ofa Dart. The DartEngine executes these instructions. Along with theconventional fetching, storing, testing, computations and branchinginstructions of conventional processors, theInteroperabilityInstructionset includes instructions to enhance thespeed of operations, carry out interoperability methodologies and exposethe capabilities and content of devices to each other. Of special noteis that there are instructions for exposing and the use of uniquecapabilities and content of devices to other device even when the otherdevices have no prior knowledge of the unique capabilities and content.

Creationism. Creationism is a method used by Darts to dynamicallygenerate Darts highly customized for a particular target device and orcommunications session and or purpose. Instructions in theDartInstructionSet exist for programmatic generation of Darts from partsof the running Dart itself and any information that can be collected orcomputed by the running Dart.

Interoperability Engine/DartEngine. The DartEngine is software and orhardware used to execute the instructions of Darts on a device and carryout their intended purpose. The DartEngine and the device specificDartPlayer, in which it is encapsulated, provides the common executionand DartRuntime environment which allows Recruitment and Renditioning toestablish efficient teams of devices and spread their code, data andcontent as best to carry out the intended purpose of Darts.

Interoperability Device Enabling. Interoperability Device Enabling isthe process of turning a conventional device into a highly interoperableDartDevice through the porting of a DartEngine as part of a DartPlayer.In addition, implementation of the Hardware Abstraction Layer needed toaccess the device specific information, capabilities and content of thedevice is also required. At least one communications protocol must beimplemented before a device with a DartPlayer becomes a DartDevice.

Interoperability Security Model/DartSecurity. DartSecurity is a systemfor providing the infrastructure needed for protecting the integrity ofa device and its content from malicious or accidental damage.

Social Synchronization Interoperability Method/Dart SocialSynchronization. Social Synchronization is an efficient and easy toadministrate method for synchronizing specific sets of data and oroperations across any number of devices and protocols without the needfor every device to contact a master device, or for any device to act asa master. SocialSynchronization of devices and content is similar to theway humans share information and tasks and is an advantageousalternative to mastered synchronization techniques most often used inthe current state of the art.

Social Security Interoperability Model/Dart SocialSecurity.SocialSecurity is a particularly simple to administrate method forforming webs of security between teams of possible intermittentlyconnected devices. SocialSecurity works in a similar way to how humansoften come to trust one another. The foundation for SocialSecurity isthe use of SocialSynchronization to spread unique ids generated usingthe DartSecurity system along with the allowed access rights whichtravel transitively from device to device. Devices which have neverdirectly communicated will often find that they are part of a team ofdevices which are allowed to interoperate with certain access rightswithout any need for further gathering permissions.

Interoperability Device/DartDevice. A DartDevice is a highlyinteroperable device by virtue of its running a DartPlayer containing aDartEngine and at least one communications protocol for connecting toother DartDevices.

Interoperability Platform/DartPlatform. The DartPlatform is any set ofDart methodologies which can carry out the specification, generation,intelligent teaming of DartDevices and facilitate the spreading andrunning of Dart interoperability applications across one or moreDartDevices.

Virtual Pointers. VirtualPointers is a methodology for providingprogrammers with a simple and efficient way to access and use of one ormore independent data address spaces in a single program.VirtualPointers used by software programs can adapt their use of mainmemory and storage devices to run efficiently on devices with differingsizes and speeds of a fast but small main memory and larger but slowerstorage.

BRIEF DESCRIPTION OF THE DRAWINGS

The Detailed Description of the Illustrative Embodiments should be readin conjunction with the appended figure drawings, wherein:

FIG. 1 is an illustration showing the situation in which getting Ndevices to interoperate or work together generally requires on the orderof N-squared adaptations.

FIG. 2 is an illustration showing the complexity of getting eightdevices to work directly together and that it takes on the order offifty-six adaptations and requires fifty-six different testconfigurations to test and verify operation.

FIG. 3 is an illustration showing a block diagram of the main componentsand sub-components of an embodiment of the invention.

FIG. 4 is an illustration showing an exemplary typical DartDeviceoverview according to an embodiment of the invention.

FIG. 5 is an illustration showing an exemplary embodiment of arecruitment procedure in the form of a flow chart.

FIG. 6 is an illustration showing a diagram illustrating an example ofthe inventive Recruitment method for extending and sharing a Dartapplication (such as a slide show Dart application) running on a firstDartDevice with a second DartDevice which originally knows nothing aboutthe originating first DartDevice or the Dart application.

FIG. 7 is an illustration showing a diagram illustrating an example ofRecruitment for remote printing from a slide show application running onan originating DartDevice on another DartDevice which originally knowsnothing about the originating DartDevice or the slide show application.

FIG. 8 is an illustration showing a diagram showing an exemplaryembodiment of a print picture application expressed as a Dart runningacross a team of three devices each of which is running a differentrendition of the originating Dart.

FIG. 9 is an illustration showing embodiments of synchronization andserialization of a Dart application running across a recruited team ofcooperating devices.

FIG. 10 is an Illustration of how Recruitments makes a group of devicesact as one device and limits the effort to achieve interoperability of Ndevices so that the effort is simply proportional to N.

FIG. 11 is an illustration showing a block diagram of an embodiment ofthe main DartFramework objects and their relative position in the classhierarchy.

FIG. 12 is an illustration showing a block diagram illustrating anapplication development device and its static and procedural componentsused to produce a Dart application.

FIG. 13 is an illustration showing a block diagram illustrating thestructure of an embodiment of the structure of the Parts component of anembodiment of the DartFormat.

FIG. 14 is an illustration showing a block diagram of the structure ofan embodiment of the Part Table component of the DartFormat and theformat of the Part Table Record components of the Part Table, whileseparately depicting the structure of an embodiment of a DartProcedure.

FIG. 15 is an illustration showing a flow chart diagram of theDartRuntime processing for a pass through the hierarchy of Gizmo derivedobjects in a Dart application.

FIG. 16 is an illustration showing a flow chart diagram illustratingaspects of Process Event processing portion of the DartRuntime.

FIG. 17 is an illustration showing a block diagram illustrating theconnections between DartDevices used by an embodiment of a Dartapplication which starts on one DartDevice and then extends itself usingRecruitment to run across other DartDevices which then exchange messagesin the form of the Event data structure shown to coordinate theiractivities.

FIG. 18 is an illustration showing a block diagram illustrating thehierarchical processing arrangement and order of execution of Gizmoprocessing units used in an exemplary embodiment of a slide show ormedia display application.

FIG. 19 is an illustration showing an embodiment of Dart ApplicationLevel Error Recovery according to one embodiment of the invention.

FIG. 20 is an illustration showing a flow chart diagram illustrating theBuiltinInstruction and OEM_BuiltinInstruction processing carried out bythe DartEngine.

FIG. 21 is an illustration showing a hypothetical NeutrinoDetector/Mobile Phone example embodiment showing how one device canexpose its unique capabilities to devices with no prior knowledge ofthese unique capabilities.

FIG. 22 is an illustration showing a block diagram illustrating anexemplary embodiment of a DartPlayer and the two main components of aparticular implementation of an embodiment of a DartEngine, namely aportable component, and the non-portable Hardware Abstraction Layercomponent.

FIG. 23 is an illustration showing a flow chart diagram of the main loopprocessing flow of a particular embodiment of a DartPlayer.

FIG. 24 is an illustration showing a flow chart diagram of theprocessing that occurs during a call to the DartEngine initializationfunction by the DartPlayer.

FIG. 25 is an illustration showing a flow chart diagram illustrating theDartRuntime instruction processing implemented in the DartPlayer.

FIG. 26 is an illustration showing a flow chart diagram for file systeminstruction processing of a DartPlayer.

FIG. 27 is an illustration showing a block diagram illustrating thecomponents of the non-portable Hardware Abstraction Layer component ofan embodiment of the DartEngine.

FIG. 28 is an illustration showing a diagram illustrating thenon-portable DartDevice specific portions of the file system which mayprovide access to files from the Dart but does not provide any mechanismfor access to files that are not considered part of the running Dart'sprotective Sandbox.

FIG. 29 is an illustration showing a block diagram of an embodiment of aDartDevice's DartSecruity components.

FIG. 30 is an illustration showing an embodiment of a SocialSynchronization contact list example.

FIG. 31 is an illustration showing a block diagram of an exemplaryDartDevice and its SocialSecurity components, and also block diagramsshowing the before and after teaming SocialSecurity contents of an oldand a new DartDevice.

FIG. 32 is an illustration showing an example Dart Virtual Pointersbeing used to access a data element at a specific virtual pointeraddress.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS OF THE INVENTION

1. Overview and Introduction

In one aspect, the present invention relates to methods of and systemsfor enabling devices to efficiently share a diverse set of content,control, resources and applications between similar and more importantlydiverse and dissimilar sets of devices and systems. Aspects of theinvention are embodied as the “DartPlatform.” The name, “Dart,” isintended to convey that the applications, data, and/or other hybridforms of processes, procedures, data, intentions, and other informationin the broadest possible sense are combined into Data that's smART, andalso to denote the manner in which complete intelligent packages ofprocedures, content and data in the form of Darts, DartProcedures, andDartParts literally dart (or are communicated) between devices toachieve a high degree of simplicity, efficiency, robustness security andinteroperability across a diverse set of devices, device resources anddevice capabilities.

Furthermore, in the description contained herein, it will be appreciatedthat the term Dart means or refers to one particular embodiment of theinvention and that the term Interoperability is used as the generic termfor aspects of the claimed invention of which a Dart is a particularform of interoperability.

The invention introduces many new technological system, device, method,and computer architecture and programmatic features that establish newparadigms not heretofore described in the literature. At least in partbecause the technical and computer terminology does not yet providecompact terms that by themselves may fully identify the innovativeelements from conventional elements, this description frequently usesthe term “Dart” or other “Dartism” as a prefix or qualifier for anotherterm, such as used in the phrases Dart Procedures, Dart Parts, and thelike. In some instances, the two terms are connected such asDartProcedures, DartParts, DartDevice, and the like. Furthermore, itwill be appreciated that even more compact forms such as device, part,or procedure may be utilized in describing aspects of the invention. Theintended meaning should normally be apparent from the context of thedescription; however, it should be appreciated that capitalized singleword forms such as “DartProcedures” are equivalent to multiple wordforms such as “Dart Procedures”, “Dart procedures”, or even “procedures”when describing an aspect of the invention. This applies to other“Dartisms” as well, such as Dart Parts, Dart Player, Dart Platform, andthe like.

Darts in their most general form are not simply data and not simplyprocedures or programs and not simply content, though Darts may be anyand may combine elements of all. “Darts” can be thought of as a specialintegrated digital binary package of software, code, data, and/orcontent along with the code data and content that can be used tointelligently assemble, save and distribute itself or parts of itselfacross devices to carry out the intent of an interoperabilityapplication. The invention results in simple, efficient, reliable andsecure interoperability between both homogeneous and heterogeneousdevices.

Unfortunately the contemporary paradigm for computing and informationsystems and devices has become so entrenched in common practice ofseparating and distinguishing operating system (OS) components, devicedriver, computer program application programs, and user or other datacomponents, that some commonly accepted terms used in the computerscience arts do not strictly apply to elements of the present invention.Therefore to the extent possible, common computing and computer scienceterms are used when their meaning is appropriate even if not exact, andvarious “Dart” expressions and terms are applied when a more specificmeaning is intended to avoid use of generic language with manyqualifiers.

Some components, features, and advantages of embodiments of theinvention are first described to orient the reader to the Dart Platform,system, method, and computer program elements. These and othercomponents, features and elements are then further described in theremainder of the detailed description. It will be appreciated that notall components, features, or advantages provided by every embodiment ofthe invention can readily be listed in a few paragraphs, therefore thedescription below is exemplary of components, elements, and advantagesfound in some embodiments, including some optional but advantageouscomponents and features, and should not be taken as a limitingdescription. It will be apparent that embodiments of the invention mayprovide and use or not provide or use some or many of the featuresdescribed herein, and that other embodiments will provide and use manyor most if not all of the features and components described here.

Existing methodologies for operating systems, program formats,programming tools, and communications protocols were developed largelyfor a world of ever more powerful general purpose computers which haveaccess to ample reliable power supplies and reliable, high-speed, alwayson, communications protocols. Furthermore, networks of computers wererelatively static and rarely had to be configured or reconfigured towork together. Also it was assumed that there would be a knowledgeablepool of human system administrators available for installing,configuring, reconfiguring, updating, fixing and otherwise maintainingcomputers, networks, operating systems and programs.

These existing methodologies are ill suited as commonly applied to thenow quickly evolving world of special purpose devices which often run onbatteries, have limited computing resources, communicate through lowerspeed unreliable wireless or power-line protocols and need to bedynamically reconfigured constantly to work with other devices, many ofwhich are portable and only intermittently connected. Despite theincreasingly dynamic configuration needs and diversity of devices andtheir capabilities which must work together, it is often highlydesirable for these devices and associated software, to be used andmaintained by people who are not knowledgeable as systemsadministrators.

The DartPlatform includes a set of new inventive methodologies andcomponents of a software, and optionally hardware, eco-system designedspecifically around the characteristics and needs of the now quicklyevolving world of special purpose and communicating devices.

According to one embodiment of the invention, the inventive DartPlatform (DP) may advantageously include the following components:

-   -   1. DartInstructonSet—An interoperability instruction set    -   2. DartEngine—A portable interoperability engine    -   3. DartFormat—A file or bit-image format to encapsulate any        possible combinations of data, content, codes, procedures,        semantics or other information that may be necessary to run,        save, optmize, copy and/or share the data, content, code,        procedures, or other information optimally (or near optimally)        on any device or subsystem that contains both a DartEngine and a        mechanism for delivering DartFormat information (usually in the        form of a set of digital binary such as in bit file(s) or        bit-image(s) to the device for processing by the DartEngine.    -   4. Dartools—A set of software tools for creating Dart Format        bit-images and files. Dart Tools may include a DartCompiler, a        DartLinker, and a DartMasterPlayer, as well as other tools, each        of which are described in greater detail herein below.    -   5. Dart or Darts—The package of bits, bit image and/or file        instances created by the Dartools for processing by a device        containing a DartEngine that conforms to the DartFormat file or        bit-image format. The DartFormat may encapsulate combined        data/content/codes and the semantics necessary to run, save,        optimize, copy and/or share the data/content/code optimally on        any device or set of devices that contain both a DartEngine and        a mechanism for delivering DartFormat bit-images to the device        for processing by the DartEngine. Darts is the plural form of        Dart.    -   6. DartProcedure(s)—Self-contained lightweight procedure(s) and        data made up of sequences of instructions from the        DartInstructionSet combined with data images and data image        values the instructions operate on.    -   7. DartPlayer—A software program designed to run on a particular        subset of devices which employs a DartEngine to process Darts.    -   8. DartMaster—A Dart intended to be further optimized into one        or more efficient and/or specialized Darts or Dart by processing        the DartMaster with a special Dartool called the        DartMasterPlayer.    -   9. DartFramework or DartObjectFramework—A set of source code        which defines the data and methods for a set of base classes for        use in building Darts. The DartObjectFramework assumes a certain        initial execution point, initial object data pointer, and set of        structures and semantics that govern the order that code and        data and input data and code are processed. This embodies the        device intra-device runtime portion of the DartRuntime which        also extends the intra-device runtime into an interdevice        runtime which synchronizes and serializes the runtime activities        across any number of teamed devices.    -   10. DartRuntime—The DartFramework or DartObject Framework        advantageously assumes a certain initial execution point,        initial object data pointer, and set of structures and semantics        that govern the order that code and data and input data and code        are processed. This is referred to as the intra-device portion        of the “DartRuntime.” There is also an inter-device runtime        which assumes that event structure instances which drive the        governs the synchronization and coordination of activities and        data exchanges amongst teamed devices are automatically        serialized and synchronized between event queues one of which is        maintained by the DartEngine of each device. Together the        intra-device DartRuntime and inter-device runtimes together        provide for a simple to use but highly robust and effective        system for carrying out the intent of a Dart where possibly        different renditions of the Dart are each running on a set of        teamed devices.

Among other aspects, this invention is designed to improveinteroperability between and among devices of any type and to solve thefollowing problems and others in interoperability amongst intermittentlyinterconnected or always interconnected devices, systems, or subsystemswithout limitation:

-   -   1. Adaptation and optimization of display, control, code, data        and functionality when moving content, applications and controls        between dissimilar or heterogeneous devices.    -   2. Remove the need for the device user to have to think about        programs, content formats, drivers and/or files, when all they        want is content that works (and preferably works as well as        possible) no matter where the programs, files and or content        goes.    -   3. Remove the need for the user to have to specify or even know        about the type of device connection(s) or communications used        between one or a plurality of devices.    -   4. Allow for the simple and efficient sharing of content,        controls, and/or resources between all enabled devices.    -   5. Enable any device or system, from expensive and complex        computer-aided tomography (CAT) scanners down to (thin        processor/memory) light switches or other simple devices to make        available, document and grant access to their functions,        resources, capabilities and needs to other connected devices.    -   6. Bring the overwhelming development effort necessary using        conventional non-procedural approaches to make some or all of        the above possible down to a level that can be easily achieved        with a dramatically lower number of development projects and        costs.    -   7. Be lightweight (in terms of code size, execution logic, and        memory) enough to be suitable for low-end, cost-constrained,        and/or power constrained devices.    -   8. Insure that any number of DartDevices can seamlessly        interoperate with all DartDevices, in most any manner that makes        sense, such as for example by including intimately entwined        native support for one or any combination of:        -   (a) Dynamic multimedia rich interfaces whenever the device            hardware can support it, and hierarchically less rich            interface if the device hardware does not support it;        -   (b) Device power management;        -   (c) Device discovery;        -   (d) Service discovery;        -   (e) Resource discovery;        -   (f) Isolating the Dart applications from needing to know the            details of the various communications physical and protocol            layers;        -   (g) Isolating the Dart applications from needing to know the            details about physical display formats;        -   (h) Isolating the Dart application from needing to know            details about maintaining synchronization with applications            running across multiple similar or dissimilar devices;        -   (i) Requesting, retrieving and running optimized control            panels to run locally, but actually control the device from            which the control panels were retrieved; and    -   (j) Loading, running and optimizing separately produced Dart        content as child Dart extensions of the intra-device DartRuntime        environment of one or more parent Darts.    -   9. Eliminating the need for programs, data or content to        pre-exist on any of the devices other than the device        (initiating device) that initiates the interoperability        application embodied in a Dart on the originating device.    -   10. Allowing devices to efficiently share their resources based        on all or some subset of devices capabilities and limitations,        with no need for any device to be master or slave.    -   11. Allowing applications' code, data, content and hybrids of        these (as embodied in Darts) to dynamically spread themselves to        connected devices in a manner that allows the connected device        to save the application for use and further spreading to other        devices ad infinitum even after the original or initiating        device is no longer connected.    -   12. Improving reliability of multi-device operations by        encapsulating all the code, data and content of an        interoperability application in a single package (or set of        packages) which then spreads itself across other devices as        required, eliminating the problems associated with mixing and        matching separately generated and distributed application or        protocol software implementations, code, data or content that        otherwise would have to be separately generated and/or        distributed to each device.    -   13. Accomplishing all the above in a simple, reliable and secure        manner.

With reference to FIG. 8, there is shown an example PrintPicture Dartrunning on a cell phone which has recruited a network attached storagedevice to use as a source of pictures and a Printing device to serve asthe destination for carrying out the printing of the pictures. All thedevices are assumed to contain a DartPlayer that has been ported to eachdevice.

The Dart on the cell phone contains the three renditions (R1, R2 and R3)which are each capable of serving as individually executable images whenrun on a DartPlayer. Rendition R3 is running on the cell phone andpreviously carried out the recruitment of the other two devices bysending DartProcedures to execute on the candidate devices whichdetermined that the particular Network Attached Storage device wouldfunction as the best source for pictures, and that the Printer devicewould best function as the destination for printing pictures.

Rendition R3 on cell phone then generated the image for a Dartcontaining just the rendition R1 which contains event processing codefor identifying and retrieving pictures from the Network AttachedStorage (NAS) device. The Dart containing rendition R1 is then sent tothe Network attached Storage device as part of a RUN_DART type event.When the RUN_DART type event is processed by the DartEngine on the NASdevice, rendition R1 is loaded and begins execution which will processany synchronization events requesting information or pictures stored onthe NAS device. Similarly, rendition R2 which handles the printing ofpictures that are part of PRINT_PICTURE type events is formed and sentto run on the chosen Printer device.

Note that the three renditions were all generated from the samePrintPicture Dart that was originally only resident on the cell phone.This ensures a high likelihood of compatibility since the application isin effect talking to parts of itself rather than independentlyimplemented, ported, and distributed component required for conventionalinteroperability methods.

Note further that the renditions also share code, data and/or contentand understand an interrelated set of event types specific to thePrintDart application. The R3 rendition was able to spread portions ofthe PrintPicture Dart intelligently to DartDevices even though it had noprevious knowledge of the other two DartDevices, and which devices hadno previous knowledge of the cell phone or the PrintDart.

Now rendition R3 running on the cell phone can signal events to controlthe selection and printing of pictures between the recruited NAS andPrinter devices across an event driven DartRuntime now established onand amongst the three devices.

Note additionally that any combination of protocols may be employedaccording to common protocols of the cell phone and the NAS device andindependently chosen for use any common protocols of the RecruitedPrinter device. In addition, since all devices contain a portedDartEngine as part of the DartPlayer, the devices can interoperate evenif they are running different operating systems on different processors.

As the invention involves a system with a large number of elements thatmay have closely tied complex interrelationships, some of the elementsare first described in overview manner so that some understanding as towhy an element may be present and how the element works and interactswith other element. Then each element will be described in even furtherdetail with due regard for its interaction with other elements. Makingthe dramatic improvements to the state of the art in interoperabilityembodied in the invention required the creation of a largely new type ofsoftware ecosystem akin to the move from data passing to functions tothe object oriented methodologies now in widespread use. In order tomore easily describe the inventive, system, and data and code structuresit is useful to first introduce some new terminology to describe thetheory, concepts, characteristics and components of embodiments of theinvention.

In one embodiment, the invention itself embodies the following nineenabling methodologies, some of which may be optional but areadvantageously included: Recruitment, Interoperability Instruction-set,Renditioning, Creationism, Vertical Layering, Linear Tasking, SocialSynchronization, Social Security, and Virtual Pointers.

Device “Recruitment” may include a device interaction model andassociated structure and methods and is an advantageous alternative tothe existent client/server and peer-to-peer models. The“Interoperability Instruction-set” is portable engine based instructionset which provides a common efficient executable environment acrossdevices and inventively provides a programmatic mechanism for exposingthe unique capabilities of a device to other devices “Renditioning”refers to structures and methods to enable easily tested and efficientadaptation of application, data, content, and/or other information. Insome aspects renditioning may include efficiently segmenting groups ofcode, data and content into independently executable images,“Renditions,” to be intelligently generated and distributed across aplurality of devices. “Creationism” refers to structures and methods toenable the dynamic efficient generation and distribution of application,data, content, and/or other information in differing forms acrossconnected and intermittently connected devices. “Vertical Layering”enables efficient and effective implementation of features which bytheir nature involve close cooperation at the tool, application,framework, engine and instruction-set levels. “Linear Tasking” enables asimple deterministic flow of processor control and device resourcesbetween processing units of the devices, where the processing units caneasily be reconfigured into a single hierarchy which includes processingunits compiled into a Dart, separately compiled Darts, and even Darts orRenditions running on other devices. “Social Synchronization” refers toan inventive efficient system and method for synchronizing data orcontent across any number of devices with minimal or no userinvolvement. “Social Security” refers to an inventive and efficientsystem and method for easily establishing and maintaining groups ofdevices with the proper authorization to interoperate with minimal userinvolvement. These and other components may be implemented in the formof computer program code segments that include executable instructionsfor execution within a processor of a device. Furthermore, elements ofthese components may be embodied in hardware and or a combination ofhardware with software and/or firmware and or microcode.

Another enabling methodology that may optionally but advantageouslyprovided is referred to as “Virtual Pointers”, and provides a variant ofand significant improvement over virtual memory that has severaladvantageous properties, including for example:

-   -   (a) Multiple large independent address spaces    -   (b) Application specific control over the number of real memory        pages.    -   (c) Device specific control over real memory page sizes to match        storage device performance characteristics.    -   (d) No need for the programmer to predict or administrate the        amount of memory needed for data structures or lists.    -   (e) Automatic saving of data in a form independent of page sizes        or number of pages.    -   (f) Effectively caches data from a larger and possibly slower        data store with a minimal amount of precious fast RAM allowing        applications to run as if they have much larger RAM memories        than they physically do.    -   (g) Simple and efficient base infrastructure for indexed        database operations where the data and indexes are kept in        different virtual pointer address spaces.

Having now described many of the methodologies, components, and featuresof the invention in overview manner, attention is now directed todetailed descriptions of embodiments of the principle methodologies,structures, and methods. It will be noted that many of the proceduresand methodologies may be implemented by one or more computer programs orcomputer program products that may be executed on general or specialpurpose processing logic, such as micro-controllers, processors,microprocessors, central processing units (CPU), or other processinghardware or logic that is capable of executing or operating on computerprogram code whether in the form of software, firmware, or a combinationof the two.

Section headers where provided are merely intended to direct theattention of the reader to portions of the specification where oneparticular aspect or methodology is described, but it will beappreciated that aspects of all the methodologies and structures aredescribed throughout the specification and in the drawings and claims,and that no limitation should be implied by inclusion or exclusion ofany aspect of the invention within a sub-headed section.

II. Recruitment Interoperability Model

“Recruitment” includes a device interaction model and methodologyembodied throughout the implementation of the invention. It is anadvantageous alternative to the Client/Server and Peer-to-Peer deviceinteraction models and methodologies used in the current state of theart. The recruitment model and methodology utilizes a common proceduralenvironment that runs on all devices that are to interoperate or beinspected for resources in contemplation of a possible interoperation.In one embodiment of the invention, this common procedural environmentis provided by an instruction set, such as the Dart instruction set(e.g., the DartInstructionSet), or an instruction set by any other namethat fulfills the requirements for the common procedural environmentdescribed here or its equivalent.

With reference to FIG. 5 there is illustrated a flow chart diagram of anembodiment of a recruitment procedure that provides a method for asoftware application to recruit and effectively later run across aplurality or team of devices much as if they were one device with thecombined resources of all the devices. Reference is also made relativeto an example of recruitment for a shared slide show 10000 FIG. 6 and anexample of remote printing of one or more slides from a slideshow 20000FIG. 7 Device recruitment (or more simply, “recruitment” or“Recruitment”) is performed. Recruitment provides a method for asoftware application to recruit and effectively run across aconstellation, team, or other plurality of devices and/or systems muchas if-they were one device with the combined resources of all thedevices.

A resource may be virtually any hardware, software, firmware,communication capability, configuration, data or data set, capabilitypossessed or accessible by a device or system. A resource may forexample be a processor, a CPU, memory, a list of contacts, a soundoutput device, a display type, DARTs, pictures, or any other procedural,structural, or information item without limitation. A capability may forexample be computer code to sort a list, hardware or software to decodean mpeg file, communicate with Bluetooth devices, or the like. Therecruitment procedure has the intelligence (by virtue or its programmingand supporting frameworks, platforms, engines, and the like describedherein, to independently of the initiating device, make complexdeterminations as to the suitability of the recruitable or recruiteddevice to carry out the intent of the application which sent theprocedure, or some portion of the intent of the application.

In one embodiment, the initiating device first sends or broadcasts amessage in the form of an inspection procedure or procedures in a commonexecutable form, from itself as the source or initiating device, to anynumber of reachable devices over any number of communications protocols,see FIG. 6 10011 and FIG. 7 20011. The initiating device forms and sendsthis message in an attempt to find other devices with needed resourcesor capabilities, a procedure or plurality of procedures structured,coded, or otherwise implemented in the common procedural environment issent, transmitted, broadcast, or otherwise communicated over aconnection or plurality of connections to other devices (known orunknown at the time of the communication) which also contain the commonprocedural environment. The initiating device need not know what devicesmay be reachable when it sends or broadcasts the message. Whether otherdevices are reachable may for example depend on the communicationschannels and/or protocols that the initiator device has as compared withthe possible set of candidate recruited devices.

This source device is alternatively referred to as the initiator devicebecause it initiates the interaction (see FIG. 6 10100 and FIG. 720100), the source device because it may be the source of a recruitmentprocedure, or the recruiter device because it is the device that isattempting to recruit other devices which may themselves be referred toas recruited devices, destination devices, target devices, or simplyother devices than the initiator device.

If for example, the initiator device includes a Bluetooth™ wirelesscommunications link, an Infrared communications link, and an IEEE802.11(a)(b) or (g) communications link and broadcasts the message overeach of these channels, only candidate recruitble devices thenconfigured to receive communications over these communications links mayreceive the recruitment message. Of these, only those devices thatprovide the operating environment and understand and can execute theinspection procedure will respond. Note that even among such possiblyresponsive devices, a user of the otherwise recruitable device mayselectively or globally block such recruitment or interrogationinspection procedures, for example according to security settings.

Second, the inspection procedure then executes its instructions on eachdevice that responded and was found, to identify needed resources andcapabilities and/or interrelated sets of resources and capabilities ofthe device, or those available to the device for carrying out theapplication or portion of the intent of the application. In oneembodiment, this inspection is performed with the use of an instructionset profile or get profile instruction, such as for example the Dartinstruction set profile instruction (e.g. DartInstructionSetPROFILE_INSTRUCTION) which results in an access or call to the HardwareAbstraction Layer (HAL) of the candidate recruitable device to accessinformation about the particular device and its resources andcapabilities as stored or computed in the device specific HardwareAbstraction Layer (See for example the Hardware Abstraction Layer HAL652 of FIG. 27.

Third, the inspection procedures return procedures, data, content orother information to the initiating device over the communicationchannel and protocol indicating that it received the message (see FIG. 610012 and FIG. 7 20012). The responding device may either identify andsave an indication of the communication channel and protocol over whichis received and optionally the identity of the device from which therecruitment message was received, or may broadcast a response that willlikely be received by that initiator device.

In some embodiments, the responding device only answers the initiator'squery as to the availability of a particular resource or set ofresources (such as a color printing capability), while in otherembodiments, the inspection procedure may identify and respond with acomplete list of features, capabilities, and resources. Usually, asimple yes I have the needed resource is preferred (response is a singlebit or byte or word), so that the size of the communication isminimized. Codes identifying one or more types or class or of resourceor resource subclass may alternatively be used. In one embodiment, theinspection procedures return information to the source device as to theresources and capabilities of the device they ran on. This informationcan for example, take the form of a static data structure, a procedure,a Dart, a free-form tagged language, or any other form of information.See return FIG. 610012 of yes response, as well as processor speed inMIPS and preferred display resolution of the recruited dart device FIG.6 10200 in the slide show example 10000 of FIG. 6 and return of FIG. 720012 of yes response and preferred printer resolution of the recruiteddart device FIG. 7 20200 in the slide printing example of FIG. 7.

Fourth, the application code on the initiating device collects all ofthe returned information possibly including procedures, data, content,or other information from all of the responding reachable devices, andexecutes any procedures received and inspects them in order to determinehow to make use of the reachable devices and their resources tocarry-out the intent of the application.

Fifth, the application specific code on the initiating device spreadscode, data, content, or any other information to each of the reachabledevices as needed according to the determinations in the fourth step(see FIG. 6 10020, 10021 and FIG. 7 20020, 20021). The application code,data, content, or other information may be sent in single common form ormay be customized according to embodiments of this invention, such as byusing methods described relative to Renditions and Creationism, oraccording to other methods and techniques.

Sixth, optionally but advantageously recursively have code, data andcontent now spread across the initiating and initially reachable devicestogether with the application code, data and content resident on thereachable devices spread farther recursively as necessary using thefirst through fifth steps on the initial set of reachable devices, nowacting as initiating devices (secondary or subsequent initiatingdevices) as necessary to extend the application as needed to otherreachable devices until all desired devices and resources needed ordesired to carry-out the original application intent have been reached,effectively forming a complete team of devices and their associatedresources. It will be appreciated that the secondary or subsequentrounds of recruitment may not be necessary if devices having neededresources were identified in the first round. On the other hand, thesecondary or subsequent round of recursive recruitment may be necessaryif required or desired resources cannot initially be found. Therecursive recruitment also increases the possibility of a first roundrecruited device operating as a bridge or translator so that theoriginal initiator acting through a first round recruited device cancommunicate with a second round recruited device (acting for example, totranslate a Bluetooth communication to a hardwired network connection).

Seventh, have the code, data and content now distributed from and amongthe team of devices according to the needs of the initiating applicationperform the required operations and resource access, to carry out theintent of the originating application by executing code and exchangingwhatever code, data, content, or other information that is necessary ordesired to carry out or coordinate the operations that are to performedacross the said team of devices to carry out the intent of theinitiating application.

This step of distributing executable code, data, and content may be anongoing operation of the application itself—up to this point the processwas primarily focused on were forming the team spreading data, code andcontent to establish the members of the team with what they need to dotheir part of the application. The process continues to carry out theintent of the application or task. In the example of a slide show,slides (really digital images or data sets) are being added to a jointslide show, or slides may be flip or sequentially displayed on all thedevices for viewing on all devices. Procedures for continuing theinteroperation beyond this initial recruitment phase are describedelsewhere herein.

This completes the initial recruitment phase and provides an opportunityfor the recrutor (initiator) and recrutees (other teamed devices) tointeroperate and share resources as described elsewhere herein an as maybe appreciated as shown in FIG. 6. 10031, 10032, 10033

The application's operations across devices may then be synchronized byhaving the Events 800 (see FIG. 17) which drive the application'sprocessing as described herein elsewhere, and advantageously throughwhich all input to the application is obtained, placed in the queue ofall recruiting and recruited devices that are marked as beingsynchronized for its Event type. Events and event queuing are describedin greater detail hereinafter, including relative to FIG. 17.

With reference to FIG. 17, Events 800 may be data structure instanceswith the fields shown in 800 FIG. 17, including in this embodiment, anevent type 801, event parameters 802, and an event related file 810 thatmay include procedure 811, content, 812, and/or Dart 813. These Events800 may provide input, communicate data and semantics, and carry outsynchronization functions as they flow through a runtime, such asthrough a DartRuntime 8000 (See FIG. 15), Gizmo hierarchy 10000 (SeeFIG. 18), or between DartDevices 400 along the lines labeled 451-x (SeeFIG. 17).

Events 800 may carry references to files 810 for any kind, kinds, oramount of data. References may preferably be made to open files.Typically the files contain one or more of DartProcedures to be run onanother DartDevice, complete Darts to extend the reach of a Dartapplication across another Dart Device, a Dart containing a controlpanel to another Device, or general enumeration or input informationthat does not otherwise fit in the other parameters of the Event asshown in 800 FIG. 17. These and other Dart types and general enumerationor input information are described in greater detail elsewhere herein.Event processing is further illustrated in the DartRuntime flow charts8000 illustrated in FIG. 15 and Engine Event Processing builtin functionflowcharts 5000 FIG. 16.

A file 810 associated with an Event 800 may consider the file associatedwith the event to be separate from the event or preferably part of theevent itself since in one embodiment of the invention the structural andmethodological infrastructure that sends events automatically sends theassociated file along with the event. Therefore, in at least oneembodiment, before an event is placed in the event recipient's eventqueue, the file referred to on the sending side has been copied by thecommunication infrastructure to a file on the recipient side which canbe read using a file identifier (e.g. fileld) associated with the copiedfile image on the recipient device. Common or differentiated file namesor file identifiers may be used on the sender and recipient sides. Inthe case of a run Dart (e.g., RUN_DART type event) or run procedure(e.g., RUN_PROCEDURE) type event, when the event gets to the head or topof the queue so that it is next in the queue, the engine will cause theDartProcedure or Dart that can now be read using the fileld in the eventto execute. There may generally be events that are processed by the Dartengine itself, even where no application is currently running. In oneembodiment, driving the Event queue is always either a running Dart oran idle procedure (e.g., a Dart IdleProcedure) built into the enginewhich keeps calling the engine event processing procedure to keep thecommunications going. This is essentially a loop that keeps runninguntil there is an event to process. When power management (describedelsewhere herein) is applied, various techniques for stopping and thenwaking up or restarting this loop may be implemented.

It will now therefore be appreciated that the Recruitment model, method,and associated structures performs ad-hoc device, service, and resourcediscovery to identify needed devices, then sends enabling procedures andinformation to the devices by using an event data structure, such asEvents 800, and intelligently and effectively forms a team of devices,and then further coordinates the team of devices, again using an eventdata structure Events 800, in an effort to accomplish the original goalof the Dart 700 or application originally running on the source device.

FIG. 10 Illustrates how recruitment results in making a connected groupof devices act as if it were a single device. One of the most profoundaffects of this is that implementation and testing of newinteroperability devices and Dart based applications only requires aneffort proportional to the number of devices, N. Conventional static andprocedural approaches where there is a need to separately implement anddistribute components to all devices which need to interoperate requirean effort that grows at a rate proportional to N squared.

Other aspects of recruitment and the recruitment interoperability model,including the serializaiton and synchronization of events between therecruited devices are described elsewhere in this specificationincluding in the examples. Some particular exemplary embodiments of therecruitment interoperability model are also set forth below.

In one embodiment (1), the invention provides a method for a softwareapplication running on a source device to recruit a team of devices, themethod comprising: sending an inspection procedure operative to find adevice having a needed resource or capability to at least one reachabledevice different from the initiating source device over at least onecommunication link, the inspection procedure including inspectionprocedure instructions coded in an executable form common to both theinitiating source device and to the device the inspection procedure isintended to reach; receiving on the initiating device the returnresponse from each of the reachable devices directly or indirectly overa communication link; analyzing, by a procedure executing on theinitiating device, the received returns from all responding reachabledevices to determine a utilization plan identifying the combination ofcapabilities and resources of the initiating source device and theresponding reachable devices to best carry out the intent of thesoftware application; and distributing, by an application programexecuting on the initiating device, at least one of executable code,data, content, and/or Dart to at least one of each of the reachabledevices identified as having a needed resource or capability accordingto the identified utilization plan.

In another embodiment (2), the invention provides a method forrecruiting a team of devices, the method comprising: receiving on acandidate device and executing an inspection procedure operative todetermine if a receiving reachable candidate device has a resource orcapability needed by another recruiting device over a communicationlink, the inspection procedure including inspection procedureinstructions coded in an executable form known to both the receivingdevice and the recruiting device; and identifying the source device forthe received inspection procedure and sending a return to the sourcedevice status and information about whether the receiving reachabledevice has access to a resource or capability or set of resources andcapabilities identified as being needed by the initiating source device;and receiving in the case where the reachable device is determined bythe source device to have resources or capabilities needed to form ateam to carry out the intent of a software application at least one ofexecutable code, data, content, and/or Dart from the source, device,recruiting device, or another candidate device.

In another embodiment (3), the invention provides a method forrecruiting a team of devices, the method comprising: (a) sending, froman initiating source device, an inspection procedure operative to find adevice having a needed resource or capability to at least one reachabledevice different from the initiating source device over at least onecommunication link, the inspection procedure including inspectionprocedure instructions coded in an executable form common to both theinitiating source device and to device the inspection procedure isintended to reach; (b) receiving and executing the received inspectionprocedure on each of the reachable devices to identify if there is atleast one resource or capability of the reachable device needed by theinitiating source device; (c) sending a return to the initiating sourcedevice at least when the reachable device has access to a resource orcapability identified as being needed by the initiating source device;(d) receiving the return from each of the reachable devices directly orindirectly over the communication link; (e) analyzing, by an applicationexecuting on the initiating device, the received returns from allresponding reachable devices to determine a utilization plan identifyingthe combination of capabilities and resources of the initiating sourcedevice and the responding reachable devices to best carry out the intentof the application; and (f) distributing, by an application programexecuting on the initiating device, at least one of executable code,data, content, and/or Dart to at least one of each of the reachabledevices identified as having a needed resource or capability accordingto the identified utilization plan.

In another embodiment (4), the invention provides the method of (3)),further comprising receiving, by the at least one reachable device, thedistributed at least one of executable code, data, content, and/or Dart.

In another embodiment (5), the invention provides the method of (3)),wherein the method further comprises: interoperating with the at leastone reachable device to which the at least one of executable code, data,content, and Dart was distributed to carry out at least a portion of theinitiating device's application's intent.

In another embodiment (6), the invention provides the method of (3) ),wherein the method further comprises: with each reachable device actingas secondary initiating source devices, spreading executable code, data,content, and/or Dart, across the initiating and reachable devicesoptionally together with the application code, data, content, and orDarts resident on the reachable devices farther recursively to otherdevices reachable by previously reached and teamed devices as necessaryto extend the application as needed or as desired to other reachabledevices until all or a predetermined criteria of desired devices andresources or capabilities needed or desired to carry out the intent ofthe original initiating device application have been reached,effectively forming a larger complete team of devices; and distributingexecutable code, data, content, and/or Dart from and among the team ofinitiating and reachable devices according to the needs and desires ofthe initiating device application to perform the required or desiredoperations and resource access to carry out the intent of the initiatingdevice's application by executing code and exchanging whateverexecutable code, data, and content and or Darts are necessary to carryout and/or coordinate the operations that are to performed across theteam of devices to carry out the intent of the initiating application.

In another embodiment (7) , the invention provides the method of (3) ),wherein the initiating source device receives the return directly froman initially reached device that the initiating device sent therecruitment message to or indirectly from another reachable device viaone or more intermediary reachable devices in a serial or recursivemanner.

In another embodiment (8) , the invention provides the method of (3) ,wherein the return to the initiating source device is also sent when thereachable device does not have access to a resource or capabilityidentified as being needed by the initiating source device.

In another embodiment (9) , the invention provides the method of (3) ,wherein the return to the initiating source device is a simple parameterthat identifies that the reachable device has itself or access to (e.g.,true or logical “1”) or does not have itself or access to (e.g., falseor logical “0”) the resource or capability identified as being needed bythe initiating source device.

In another embodiment (10) , the invention provides the method of (3) ,wherein the return to the initiating source device comprises one of aDartEvent, a part of a Dart, data, content, executable procedures, aDart, a plurality of darts, and any combination of these.

In another embodiment (11) , the invention provides the method of (3) ,wherein the return to the initiating source device comprises returneddata or content identifying the resources and/or capabilities of theparticular reachable device to the initiating device over thecommunication protocols.

In another embodiment (12) , the invention provides the method of (3) ,wherein the inspection procedure includes an instruction to inspect forat least one particular identified resource or capability needed by theinitiating source device to carry out the intent of the application taskbeing performed at least in part on the initiating source device.

In another embodiment (13) , the invention provides the method of (3) ,wherein the method is implemented at least in part by a softwareapplication encoded as a computer program product recruiting andeffectively running an application across a team of devices.

In another embodiment (14) , the invention provides the method of (3) ,wherein the method is effective to couple and subsequently permitcontrol of the operations of the recruiting and recruited devices as ifthey were one device with the combined resources of all the recruitingand recruited devices.

In another embodiment (15) , the invention provides the method of (3) ,wherein the returns comprises any one of: no return, a data or contentreturn, any digitally encoded information, one or more procedures, anindication that the device will be useful, a returned event, a returnevent containing any amount of data or sets of data via its filepayload, a return procedure, a Dart, a return event that includes textnames and descriptions of the device so a human can select from a menuon the initiating device which device(s) to use, an identifier of aspecific package of at least one instance of a set of executable code,data and content residing on or capable of being generated on the sourcedevice, rendition or set of renditions most suitable to run on thedevice or a combination of any of these.

In another embodiment (16) , the invention provides the method of (3) ,wherein the at least one resource or capability is selected from the setconsisting of: (i) available resources or a particular needed resource;(ii) available capabilities or a particular needed capability; (iii) oneor more interrelated sets of resources and capabilities of the reachabledevice, or those resources and/or capabilities available to thereachable device for carrying out the intent of the application; and(iv) any combination of these.

In another embodiment (17) , the invention provides the method of (3) ,wherein the resources or capabilities include at least one of anidentified capability selected from the set of resources or capabilitiesconsisting of: an identified data manipulation software, an identifiedinformation processing software, an identified computational software,an identified picture processing software, an identified communicationssoftware, an identified communications hardware, an identified media, anidentified data set(s), an identified content, an identified program orprograms, an identified configuration information, an identifiedgraphics acceleration hardware or software, an identified storage mediumwhether temporary or not temporary (permanent), an identified printingcapability, an identified faxing capability, an identified scanningcapability, an identified user interface device whether input or outputor both input and output, an access to the resources of other deviceswith which the device can communicate and with which the other devicescan communicate in an unending chain, and any combination of two or moreof these.

In another embodiment (18) , the invention provides a method as in (3) ,wherein the inspection procedures in a common executable form comprisesat least one inspection procedure formed from a Dart compliantinstruction set (DartInstructionSet) as embodied in a Dart instructioncompatible engine (DartEngine) or any other Interoperability InstructionSet.

In another embodiment (19) , the invention provides a method as in (3) ,wherein the at least one communications link comprises any number ofcommunications links, channels, and/or protocols that comprise anynumber or set of homogeneous or heterogeneous communications protocolswhether wired or wireless, and whether permanently available orintermittent.

In another embodiment (20) , the invention provides a method as in (3) ,wherein the heterogeneous and homogeneous communications links,channels, and protocols are supported by an identified hardwareabstraction layer implementations that are parts of players running onany two or more communicating devices.

In another embodiment (21) , the invention provides a method as in (20), wherein the identified hardware abstraction layer implementationscomprises a Dart Hardware Abstraction Layer implementation that arecomponents of the DartEngine.

In another embodiment (22) , the invention provides the method in (3) ,wherein the at least one communications link and a communicationsprotocol are used to send events of a run procedure type, and an eventidentifier of the event references a file identifying the procedure torun on the reachable device.

In another embodiment (23) , the invention provides the method in (22) ,wherein the events comprise DartEvents, and the run procedure type eventcomprises a Dart RUN_PROCEDURE type event.

In another embodiment (24) , the invention provides the method in (3) ,wherein the event identifier that references a file identifying theprocedure to run on the reachable device comprises a file identifier ofthe event referring to a file containing an image of the procedure torun on the reachable device.

In another embodiment (25) , the invention provides the method in (24) ,wherein the file comprises a Dart compliant file (DartFile) conformingto the DartFormat, and the image of the procedure comprises a binarydata image of a Dart Procedure (DartProcedure).

In another embodiment (26) , the invention provides the method in (3) ,wherein the inspection procedures comprise either DartProcedures orcomplete Darts.

In another embodiment (27) , the invention provides the method in (3) ,wherein the inspection procedures are sent as the file associated withan event, and the receipt of the inspection procedure by a reachabledevice as the file associated with the event causes the inspectionprocedure to start executing on the reachable devices.

In another embodiment (28) , the invention provides the method in (3) ,wherein the inspection procedure comprises a DartProcedure.

In another embodiment (29) , the invention provides the method in (3) ,wherein resources and capabilities including base resources andcapabilities of the reachable device are determined through use of aninstruction set profile instruction.

In another embodiment (30) , the invention provides the method in (29) ,wherein the instruction set profile instruction comprises a Dartcompliant profile instruction (DartProfileInstruction) of a Dartcompliant instruction set (DartInstructionSet).

In another embodiment (31) , the invention provides the method in (3) ,wherein the inspection procedure execution within each reachable devicedetermines a best rendition of the initiating Dart embodied applicationaccording to a rendition determination rule to be sent to the or eachparticular reachable device and sends back an identifier 6 f thedetermined best Rendition as part of the returned data.

In another embodiment (32) , the invention provides the method in (31) ,wherein the Rendition determination rule is embodied in as at least oneprocedures which is adapted to perform any needed computations of anycomplexity and access any needed profile information through a profileinstruction to determine the resources, capability, and/or state of thereachable device.

In another embodiment (33) , the invention provides the method in (31) ,wherein the inspection procedure execution determines the best Renditionfrom a plurality of renditions by reference to rules that define anorder of Rendition, and checking each reachable device to determine ifall the requirements of each of the plurality of Renditions are met in apredefined order of Rendition preferences until the first Rendition inthe ordered plurality of renditions is found that meets all of theRendition's requirements.

In another embodiment (34) , the invention provides the method in (3) ,wherein the inspection procedure(s) return Darts, procedures, data, orcontent to the initiating device over the communication link using anunderstood communications protocol.

In another embodiment (35) , the invention provides the method in (3) ,wherein the returns include at least one of returned procedures, data,or content and any one or combination of: complete Darts, DartParts,DartProcedures, or DartEvents, and the one or any combination may bereturned with or without an associated event file.

In another embodiment (36) , the invention provides the method in (3) ,wherein the return includes at least one of returned procedures, data,content, and/or Dart and further optionally includes a return codeindicating an error has occurred, the error code identify either that aspecific error has occurred or that a non-specific error has occurred,and the error code optionally including information useful in correctingor communicating the particular error and/or the nature of the error.

In another embodiment (37) , the invention provides the method of (3) ,wherein the application code is at least one of a Dart, a DartProcedure,or any program of any form that can be executed on the initiating deviceor on devices which the initiating device has access to for initiatingtransfer or execution of a program and making use of the results of theexecution.

In another embodiment (38) , the invention provides the method of (3) ,wherein the application code comprises a Dart, a DartProcedure, oranother procedural format that can execute on or otherwise conveyinformation to the reachable device(s) whether or not the proceduralformat makes use of the DartInstructionSet, to be executed on thereachable device.

In another embodiment (39) , the invention provides the method of (3) ,wherein the recruited team of devices may dynamically extend the team toinclude other reachable devices or reduce the team of devices to excludereachable devices as desired during the lifetime or defined period oftime for execution of the application.

In another embodiment (40) , the invention provides the method of (3) ,wherein the distributing is accomplished through the sending of at leastone of Darts, DartProcedures, data, content, or other information, orany combination of these, that are encapsulated as part of dart events(DartEvents) whether or not the information referenced by a field in theevent is sent along or as part of the event.

In another embodiment (41) , the invention provides the method of (3) ,wherein the code, data, and content that have been distributed from andamong the team of devices according to the needs of the initiatingapplication, perform the required operations and resource access tocarry out the intent of the originating application by executing codeand optionally exchanging whatever additional or different code, data,and content that is necessary to carry out or coordinate the operationsthat are to be performed across the team of devices to further carry outthe intent of the initiating application.

In another embodiment (42) , the invention provides the method of (3wherein the application is embodied in a single binary image whichcontains all the code that is distributed to all the devices as part ofthe recruitment process.

In another embodiment (43) , the invention provides the method of (3) ,wherein the method further comprises synchronization, serialization, andcoordination of activates on the team of devices, and thesynchronization, serialization and coordination is accomplished whollyor in part by the passing or exchanging of events or DartEvents alone oroptionally with a file associated with DartEvents or events.

In another embodiment (44) , the invention provides the method of (43) ,wherein the events reference at least one file so that they effectivelycontain and incorporate by that reference a file or files having fileimages of any complexity by virtue of that reference, and these fileimages are communicated along with the event structure contents.

In another embodiment (45) , the invention provides the method of (3) ),further comprises the passing or exchanging of DartEvents events betweeninteroperating and communicating initiating and recruited devices aloneor optionally with a file associated with DartEvents or events, and theevents effectively contain files or other data or data structures of anycomplexity, by a file identifier (field) reference in the DartEventstructure, and that file images when referenced are always communicatedalong with the event structure contents.

In another embodiment (46) , the invention provides the method of (43) ,wherein the DartEvents or events are automatically copied and/orsynchronized across event queues of any number of teamed devices by theDartFramework, DartRuntime, and/or DartEngine, or alternatively by anon-DartPlatform specific event driven implementation, so thatDartEvents or events which are identified for automatic synchronizationand which are placed on an event queue by a Dart appear in the eventqueues of any number of teamed devices in an automatically serializedand synchronized manner.

In another embodiment (47) , the invention provides the method of (46) ,wherein the automatic serialization and synchronization are accomplishedby a procedure for automatic serialization and synchronization of eventdriven cooperative applications, functions or renditions running acrossa plurality of cooperating devices including an initiating device, themethod comprising: instantiating a connection manager object instance onthe originating device by an application for each inter-devicecooperative function desired; communicating a list of event types to besynchronized over all cooperating devices procedurally by theapplication to the connection manager; establishing a team ofcooperating devices having one connection manager on each device andsharing the same list of synchronization event types; and maintaining,by the connection manager, a sessions list identifying teamed devicesand event types to be synchronized; and examining, by the connectionmanager, all events to be put in the event queue, and if a particularevent examined is an event type the connection manager knows from thesessions list should be synchronized, then marking the event as an eventto be synchronized with the other sessions and placing the event in theevent queues of the devices listed in the connection manager.

In another embodiment (48) , the invention provides the method of (47) ,wherein all events to be placed on cooperating device event queues bythe any one of the event driven cooperative applications functions orrenditions are serialized by not allowing an event to be placed on theany one device's event queue until it receives acknowledgement that allcooperating device event queues to which the event has been sentdirectly by the any one device has successfully been placed on thecooperating devices' event queues.

In another embodiment (49) , the invention provides the method of (48) ,wherein all events to be placed on cooperating device event queuesreceived by any one of the event driven cooperative applicationsfunctions or renditions are serialized by not allowing an event to beplaced on the receiving any one device's event queue until it receivesacknowledgement that all cooperating device event queues to which theevent has been sent by the any receiving one device has successfullybeen placed on the cooperating devices' event queues.

In another embodiment (50) , the invention provides the method of (48) ,wherein any number of cooperating devices' event queues whether thedevices are communicating directly or indirectly through a chain ofdirectly communicating devices establishes a single serialized system ofqueued events across all cooperating devices.

In another embodiment (51) , the invention provides the method of (49) ,wherein any number of cooperating devices' event queues whether thedevices are communicating directly or indirectly through a chain ofdirectly communicating devices establishes a single serialized system ofqueued events across all cooperating devices.

In another embodiment (52) , the invention provides the method of (47) ,wherein events to be placed on the cooperating devices' queues from twoor more cooperating devices are synchronized into a single serializationof events in the cooperative devices' queues by a system where only onemaster device is allowed to place the events of the types in the list ofevent types to be synchronized so that all such events will beserialized across all cooperating devices.

In another embodiment (53) , the invention provides the method of (52) ,wherein the master device is informed of the events to issue on behalfof other cooperating devices by way of a master request event type eventwhich contains all the information needed for the master device to placethe intended event into the queues of all cooperating devices.

In another embodiment (54) , the invention provides the method of (52) ,wherein each device which has been recruited into the team ofcooperating devices by another into the set of cooperating devicesconsiders its relative master device to be the device that recruited it.

In another embodiment (55) , the invention provides the method of (53) ,wherein each device which has been recruited into the team ofcooperating devices by another into the set of cooperating devicesconsiders its relative master device to be the device that recruited it.

In another embodiment (56) , the invention provides the method of (54) ,wherein the placing of a master request event type event into a relativemaster device's queue will cause the event to be propagated from deviceto relative master device until an initiating master device which has norelative master then forms an event using the information needed for themaster device to place the intended event into the queues of allcooperating devices.

In another embodiment (57) , the invention provides the method of (52) ,wherein the designated master device can be changed by issuing to thesynchronized and/or serialized queues of cooperating devices a changemaster type event which is itself on the list of serialized events whichinforms all devices in a synchronized serialized manner that a newmaster device is to replace the current master device.

In another embodiment (58) , the invention provides the method of (54) ,where the master request event propagates thorough the queues ofcooperating devices until the new master device processes the event.

In another embodiment (59) , the invention provides the method of (55) ,where the master request event propagates thorough the queues ofcooperating devices until the new master device processes the event.

In another embodiment (60) , the invention provides the method of (53) ,where the optional file identified as part of the event specified to besent by the master request event type event, if such a file reference ispresent, is maintained on each propagating device with an identifierthat will allow it to be re-associated with the event to be sent by themaster as if it had been sent as part of the event sent by the master,this in order to reduce the amount of information that would mightotherwise have to be send as part of each event sent as the result of amaster request event type processed by the master.

In another embodiment (61) , the invention provides an initiatingapparatus comprising: a processor coupled with a memory and adapted toexecute a procedure that includes instructions for performance of anintended task; means for executing at least partially in the processorand memory for recruiting at least one recruited device different fromand external to the apparatus to participate in the performance of theintended task, the recruited device including at least the hardwareresources necessary for a version of performance of the intended task;and means stored entirely within the apparatus for supplementing theresources of the recruited device so that the hardware resources plusthe enabling supplemented resources make the recruited device fullyenabled to perform the intended task.

In another embodiment (62) , the invention provides an initiatingapparatus as in (61) , wherein the apparatus and the recruited deviceeach operate in a common procedural environment, and the means executingat least partially in the processor and memory for recruiting includesmeans for broadcasting a procedure implemented in the common proceduralenvironment over at least one connection to other devices which alsoinclude or operate in the common procedural environment.

In another embodiment (63) , the invention provides an initiatingapparatus as in (62) , wherein the means for recruiting further includesmeans for initiating the execution of the procedure on the other devicesto programmatically inspect the resources and capabilities of each ofthe other devices in order to determine if each device has a neededresource or capability to participate in a performance of the particulartask.

In another embodiment (64) , the invention provides an initiatingapparatus as in (63) , wherein the inspection is performed in eachparticular other device at least in part by accessing a device specifichardware abstraction layer information stored in or computed about theparticular device.

In another embodiment (65) , the invention provides an initiatingapparatus as in (64) , wherein the means for supplementing furtherincludes means for sending and installing enabling procedures, data,and/or content that are required to enable each device to carry out itspart of the particular task.

In another embodiment (66) , the invention provides an initiatingapparatus as in (61) ), further including means for temporally orpermanently synchronizing operations across the initiating and otherdevices, the means for synchronizing including a task event queue andmeans for maintaining the task event queue.

In another embodiment (67) , the invention provides a recruitedapparatus comprising: a set of hardware resources including a processorand a memory coupled to the processor, and computer program coderesources adapted to the performance of a set of tasks, the hardwareresources being capable or performing at least a version of aperformance of a particular task but the computer program code resourcesnot initially capable of performance of a desired version or method forcarrying out of the particular task or aspect of the particular task andmeans for receiving a communication including at least one of a computerprogram code communication and a data communication, the computerprogram code communication including supplemental computer program coderesources that render the apparatus capable of performance of thedesired version, method or aspect of the particular task.

In another embodiment (68) , the invention provides a recruitedapparatus as in (67) , wherein the recruited apparatus and theinitiating device each operate in a common procedural environment.

In another embodiment (69) , the invention provides a recruitedapparatus as in (68) , further including means for execution of theprocedure received from the initiating device to programmaticallyinspect the resources and capabilities of the recruited device in orderto determine if the recruited device has a needed resource or capabilityto participate in a performance of the particular task.

In another embodiment (70) , the invention provides a recruitedapparatus as in (69) , wherein the inspection is performed in therecruited device at least in part by accessing a device specifichardware abstraction layer information stored in or computed about therecruited device.

In another embodiment (71) , the invention provides a recruitedapparatus as in (70) , further comprising means for installing enablingprocedures, data, and/or content that are required to enable therecruited device to carry out its part of the particular task.

In another embodiment (72) , the invention provides a recruitedapparatus as in (61) , further including means for temporallysynchronizing operations across the initiating device and the recruiteddevice, the means for synchronizing including a task event queue andmeans for maintaining the task event queue.

In another embodiment (73) , the invention provides a method for formingan integrated ad-hoc on-the-fly distributed information processingsystem among a plurality of heterogeneous devices to participate in theperformance of a particular task, the method comprising: initiatingformation of the distributed information processing system by aninitiating device, the formation including broadcasting a message usingat least one communication channel and protocol with the intention toidentify and recruit other recruited devices that may possess a resourceor capability to participate in the performance of the particular task;and communicating at least one of procedures, data, and content asrequired to each of the recruited devices so that each of the recruiteddevices are made capable of carrying-out its part of the particulartask.

In another embodiment (74) , the invention provides a method as in (73), further comprising: executing at least partially in a processor andmemory of the initiating device a procedure for recruiting at least onerecruited device different from and external to the initiating device toparticipate in the performance of the intended task, the recruiteddevice including at least the hardware resources necessary for a versionof performance of the intended task; and storing entirely within theinitiating apparatus a procedure and optional data for supplementing theresources of the recruited device so that the hardware resources plusthe enabling supplemented resources make the recruited device fullyenabled to perform the intended task.

In another embodiment (75) , the invention provides a method as in (74), wherein the initiating device and the recruited device each operate ina common procedural environment, and the procedure executing at leastpartially in the processor and memory for recruiting includesbroadcasting a procedure implemented in the common proceduralenvironment over at least one connection to other devices which alsoinclude or operate in the common procedural environment.

In another embodiment (76) , the invention provides a method as in (75), wherein the recruiting further includes initiating the execution ofthe procedure on the other devices to programmatically inspect theresources and capabilities of each of the other devices in order todetermine if each device has a needed resource or capability toparticipate in a performance of the particular task.

In another embodiment (77) , the invention provides a method as in (76), wherein the inspection is performed in each particular other recruiteddevice at least in part by accessing a device specific hardwareabstraction layer information stored in or computed about the particulardevice.

In another embodiment (78) , the invention provides a method as in (77), wherein the supplementing further includes sending and installingenabling procedures, data, and/or content that are required to enableeach device to carry out its part of the particular task.

In another embodiment (79) , the invention provides a method as in (77,further including temporally synchronizing operations across theinitiating and other recruited devices, the synchronizing includinggenerating and maintaining a task event queue.

In another embodiment (80) , the invention provides a method as in (73), wherein the communication is a communication and interaction that isneither in a client-server communication interaction nor in apeer-to-peer communication interaction.

In another embodiment (81) , the invention provides a method as in (73), wherein the recruitment performs ad hoc device, service, and resourcediscovery to identify needed devices, then sends enabling procedures andinformation to the devices using events; intelligently and effectivelyforms a team of devices, and then coordinates the team of devices usingevents, in order to accomplish the goal of the Dart or application ortask originally running on the source initiating device.

In another embodiment (82) , the invention provides a method as in (73), wherein the distributed information processing system includes accessand coordinated use of some or all of the physical capabilities of thedevices.

In another embodiment (83) , the invention provides a method as in (82), wherein the physical capabilities of the device are selected from theset that optionally includes an ability to print, fax, display, rendermusic, render video, control other devices, store data whether digitalor analog on any media, manufacture goods, provide elimination, takepictures, or any other physical capabilities which can be accessed,monitored or controlled by the processing capabilities of the devices.

In another embodiment (84) , the invention provides the method of (1) ,wherein the software application running on more than one device is atleast partially performing the interoperability operations on two ormore devices with code and or data and or content that were originallypart of a single software package on the initiating device so as toenjoy a reliably of interoperability greater than that whereindependently developed and or independently distributed applicationsare used to perform the interoperability operations.

In another embodiment (85) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to recruit a team of recruited devices for interoperation, theprogram module including instructions for: sending an inspectionprocedure operative to find a device having a needed resource orcapability to at least one reachable device different from theinitiating source device over at least one communication link, theinspection procedure including inspection procedure instructions codedin an executable form common to both the initiating source device and tothe device the inspection procedure is intended to reach; receiving, onthe initiating device, the return response from each of the reachabledevices directly or indirectly over a communication link; analyzing, bya procedure executing on the initiating device, the received returnsfrom all responding reachable devices to determine a utilization planidentifying the combination of capabilities and resources of theinitiating source device and the responding reachable devices to bestcarry out the intent of the software application; and distributing, byan application program executing on the initiating device, at least oneof executable code, data, content, and/or Dart to at least one of eachof the reachable devices identified as having a needed resource orcapability according to the identified utilization plan.

In another embodiment (86) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to form an integrated ad-hoc on-the-fly distributed informationprocessing system among a plurality of heterogeneous devices to act as asingle virtual device and participate in the performance of a particulartask, the program module including instructions for: initiatingformation of the distributed information processing system by aninitiating device, the formation including broadcasting a message usingat least one communication channel and protocol with the intention toidentify and recruit other recruited devices that may possess a resourceor capability to participate in the performance of the particular task;and communicating at least one of procedures, data, and content asrequired to each of the recruited devices so that each of the recruiteddevices are made capable of carrying-out its part of the particulartask.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

III. Renditioning Adaptation and Interoperability Segmentation Model

In another aspect of the invention, system, apparatus, method andcomputer program for Renditioning are provided. Renditioning includes amethodology for segmenting interoperability applications into a set ofdiscrete execution units, exactly one of which is to be selected to runon a given device or player for a given circumstance and point in time.

In one particularly advantageous embodiment and implementation, the Dartapplication framework (DartFramework) includes Rendition objects thatserve as the top of the hierarchy of processing units called Gizmos (seeFIG. 11). For purposes of attempting to understand what a Rendition is,a single Rendition may be thought of as complete conventionalprogram—though it is not the same as a conventional program in that itmay be one of a plurality of Renditions generated and packaged togetherwith interrelated functionality and content which is necessary to carryout interoperability operations. Often no one Rendition from the setcould effectively or efficiently carry out the intent of the applicationor package of Renditions on its own. An interoperability application,such as a Dart, is an integrated collection of program parts, possiblyincluding procedures, data, content and/or resources which know how toput together the parts to form and manage dynamically and staticallyassembled programs, or Renditions, from the parts (see FIG. 3 300, FIG.13, FIG. 14). Furthermore, an interoperability application or Dartcontains the means for intelligently distributing possibly differentRenditions or set of Renditions, one set for each device in a team ofdevices, which can then communicate and cooperate as a team with otherRenditions of other devices of the team to effectively and efficientlycarry out the intent of the application.

The benefits of Renditioning include among others, the sharing of partsbetween execution units so as to limit the size of an interoperabilityapplication. Using conventional programming techniques it would often benecessary to package a set of applications so that there is one that issuitable for each target device or environment. Without Renditioning,this would often result in the need to store redundant procedures, datacontent and resources in each of the programs in the package. Anotheradvantage of Renditioning is that there can be selection procedures (seeFIG. 14 4000) associated with each Rendition to be used in theRecruitment model to intelligently select the correct Rendition to runon a target device or environment. Still another advantage ofRenditioning is that it can limit the number of adaptations possible toa small enough set that thorough testing of the interoperabilityapplication is possible, while allowing for a large enough set for thereto be at least one Rendition suitable to run well on any one device orenvironment.

Renditions can also be used to allow the Dart tools (see DartTools FIG.12 200) to automatically configure the parts into special purpose Dartsto be sent to other devices. For example, they may be prepared forprinting an image or document. Such Renditions or sets of Renditions maybe created statically by the DartTools, or dynamically through theCreationism methodology (FIG. 7 20020) described elsewhere in thisdocument which uses an interoperability instruction set, such as theDartInstructionSet, or by a combination or the two possibly inconjunction with other techniques.

Another use of Renditions is for creating a set of language or culturalversions to be selected according to the users' needs or desires. Forexample, different Renditions having essentially the same informationmay be created for the French language, Chinese language or the like;and, different renditions may provide for somewhat differentinformation, such as using photographs of Japanese models for a clothingadvertisement to a primarily Japanese audience, and photographs ofItalian models for the same advertisement to be presented to an Italianor European audience.

Still another use of Renditions is to limit the use of content thatmight be objectionable, or unimportant, or otherwise not appropriate tocertain groups, for whatever reason. Yet still another use of Renditionsis to target the use of content to certain groups or devices such aschildren or Texans. Even still another use of Renditions might be totarget different localities such as Dart Renditions that contain localweather related information, In one aspect, Renditioning provides amethod for intimately packaging parts of a set of software applicationsand associated data, procedures, content and selection criteriaprocedures into a single binary image. It also provides a database ofparts, their proprieties, their identifiers and their location in thebinary image. It further provides a mapping of parts into a plurality ofindividually executable software applications comprised of procedures,and/or data and/or content. It additionally provides a procedure or setof procedures which when executed determines which one from a set ofsoftware applications and associated data, procedures and content in abinary image are to be executed for a given device, user preferences,communications characteristics, and/or environment. It further providesa toolset which allows the automated generation and packaging of partsinto a single binary image (or if desired into a plurality of binary orother format images) according to a set of source materials. Aspects ofRenditioning also provide a mechanism for finding and accessing thedatabase of parts inside a given binary image. Example use ofdistributed renditions from within and by a single Dart are illustratedin FIG. 8 and FIG. 9.

Some particular exemplary embodiments of the renditioning adaptation andinteroperability segmentation model are also set forth below.

In one embodiment (1) , the invention provides a method for segmenting asoftware application into a set of separately executable images, themethod comprising: separating the devices to be encountered into classesaccording to their possible resources and capabilities for carrying outone or more aspects of the intent of the application; separating theenvironment or operating requirements likely to be encountered intoclasses according to the needs for distinct rendering or other runtimerequirements for carrying out one or more aspects of the intent of theapplication; specifying the data, code, content and other digitallyrepresentable resources needed for an executable image needed to be ableto carry out one or more aspects of the intent of the application oneach class of device and each environment or operating requirement;generating a utilization plan for choosing which devices andcorresponding individually assigned executable images are to be run oneach device to be used to carry out the intent of the application givena list of candidate devices, their resources and capabilities, and therequired or desired environment or operating parameters; and specifyingthe data, code, content and other digitally representable resourcesneeded to implement and carry out the utilization plan on each class ofdevice and each environment or operating requirement.

In another embodiment (2) , the invention provides a method as in (1) ,wherein the software application is intended to run across one or morehomogeneous or heterogeneous communicating devices.

In another embodiment (3) , the invention provides a method as in (1) ,wherein exactly one such selected executable image is to be selected torun on a given device with a particular environment.

In another embodiment (4) the invention provides a method as in (3) ,wherein the exactly one such executable image is to be selected and runon each device in accordance with the needs of the application withrespect to the resources and capabilities of each device and theenvironment and operating requirements.

In another embodiment (5) , the invention provides a method as in (3) ,wherein at least one of the devices is a Dart device.

In another embodiment (6) , the invention provides a method as in (1) ,further comprising executing the generated utilization plan.

In another embodiment (7) , the invention provides a method as in (1) ,wherein the software application expressed as a Dart.

In another embodiment (8) , the invention provides a method as in (1) ,wherein the separately executing images are renditions.

In another embodiment (9) the invention provides a method as in (7) ,wherein the renditions are expressed in the form of Dart Renditionspackaged by the DartTools in conformance with the DartFormat.

In another embodiment (10) , the invention provides a method as in (2) ,wherein the utilization plan of (1) is implemented in whole or part asprocedures sent to run on the candidate devices to determine theparticular class of device, its resources and/or its operatingenvironment.

In another embodiment (11) , the invention provides a method as in (10), wherein the procedures are DartProcedures comprised at least in partas instructions of an Interoperability Instruction Set.

In another embodiment (12) , the invention provides a method as in (11), wherein the Interoperability Instruction Set is theDartInstructionSet.

In another embodiment (13) , the invention provides a method as in (11), wherein the Interoperability Instruction Set is of a form that isexecutable on one or more homogeneous or heterogeneous communicatingdevices which are to run procedures.

In another embodiment (14) the invention provides a method as in (13) ,wherein the communicating devices run procedures to determine theparticular class of device, its resources, and/or its operatingenvironment.

In another embodiment (15) , the invention provides a method as in (10), wherein the procedures are expressed as Darts which are part of theapplication.

In another embodiment (16) , the invention provides a method as in (15), wherein the application is expressed as a Dart which contains otherDarts used to carry out the intent of the application on heterogeneouscommunicating devices.

In another embodiment (17) , the invention provides a method as in (1) ,wherein the recruitment method of (1) is used to send and distribute theprocedures and separate executable images to form teams of heterogeneousor homogeneous devices.

In another embodiment (18) , the invention provides a method as in (1) ,wherein parts are sharable between different separately executing imagesin different target devices and processing environments so that the sizeof an interoperability application may be limited.

In another embodiment (19) , the invention provides a method as in (1) ,wherein parts are sharable between separately executing images so thatthe amount of data to be stored in the software may be limited.

In another embodiment (20) , the invention provides a method as in (1) ,wherein parts are sharable between separately executing images so thatthe amount of data to be communicated between devices may be limited.

In another embodiment (21) , the invention provides a method as in (18), wherein parts are one of code, data, content, procedures, code sets,data sets, content, content sets, meta information on how to find andcombine the parts, descriptive text, pictures, video, images, tables ofdata, or any other unit of information or set of information capable ofbeing represented in a digital form.

In another embodiment (21) , the invention provides a method as in (21), wherein parts are DartParts and or meta information is expressed asDart RenditionsTable part, and or Dart RenditionTable parts, and or DartPart Table, and or DartTrailer.

In another embodiment (22) , the invention provides a method forsegmenting a software application into a set of separately executableimages, the method comprising: separating the devices to be encounteredinto classes according to their possible resources and capabilities;separating the environment or operating requirements likely to beencountered into classes; specifying the data, code, content and/orother digitally representable resources needed for an executable imageto be able to carry out one or more aspects of the intent of theapplication on each class of device and each environment or operatingrequirement; and generating a utilization plan for choosing whichdevices and executable images are to be run on each device to be used tocarry out the intent of the application.

In another embodiment (23) , the invention provides an apparatus forsegmenting a software application into a set of separately executableimages, the apparatus comprising: means for separating the devices to beencountered into classes according to their possible resources andcapabilities for carrying out one or more aspects of the intent of theapplication; means for separating the environment or operatingrequirements likely to be encountered into classes according to theneeds for distinct rendering or other runtime requirements for carryingout one or more aspects of the intent of the application; means forspecifying the data, code, content and other digitally representableresources needed for an executable image needed to be able to carry outone or more aspects of the intent of the application on each class ofdevice and each environment or operating requirement; means forgenerating a utilization plan for choosing which devices andcorresponding individually assigned executable images are to be run oneach device to be used to carry out the intent of the application givena list of candidate devices, their resources and capabilities, and therequired or desired environment or operating parameters; and means forspecifying the data, code, content and other digitally representableresources needed to implement and carry out the utilization plan on eachclass of device and each environment or operating requirement.

In another embodiment (24) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to segment a computer program software code application into aset of separately executable computer program software code images, theprogram module including instructions for: separating the devices to beencountered into classes according to their possible resources andcapabilities; separating the environment or operating requirementslikely to be encountered into classes; specifying the data, code,content, and/or other digitally representable resources needed for anexecutable image to be able to carry out one or more aspects of theintent of the application on each class of device and each environmentor operating requirement; and generating a utilization plan for choosingwhich devices and executable images are to be run on each device to beused to carry out the intent of the application.

In another embodiment (25) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to segment a computer program software code application into aset of separately executable computer program software code images, theprogram module including instructions for: separating the devices to beencountered into classes according to their possible resources andcapabilities for carrying out one or more aspects of the intent of theapplication; separating the environment or operating requirements likelyto be encountered into classes according to the needs for distinctrendering or other runtime requirements for carrying out one or moreaspects of the intent of the application; specifying the data, code,content and other digitally representable resources needed for anexecutable image needed to be able to carry out one or more aspects ofthe intent of the application on each class of device and eachenvironment or operating requirement; generating a utilization plan forchoosing which devices and corresponding individually assignedexecutable images are to be run on each device to be used to carry outthe intent of the application given a list of candidate devices, theirresources and capabilities, and the required or desired environment oroperating parameters; and specifying the data, code, content and otherdigitally representable resources needed to implement and carry out theutilization plan on each class of device and each environment oroperating requirement.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

IV. DartSource/Interoperability Source

Recall that the DartSource provides structure and method for specifyingall the program renditions and the code content and data needed for apackaged Dart interoperability application. DartSource extends thelanguages constructs commonly used to specify single executable programtargeted to a specific device, into a language which can also specifythe procedures necessary for intelligent recruitment of teams of devicesand the renditions needed so that there is a suitable rendition to sendto run on each recruited device to carry out that device's portion ofthe intended purpose of the application being specified.

In one embodiment (1) , the invention provides a method for specifying asoftware application package of digitally encoded data, code andcontent, along with meta information in the form of data, code andcontent needed to carry out an intended purpose (intent) on one or moreconnected or intermittently connected devices; the method comprisingexpressing in an interoperability software programming language one ormore or any combination of the following: (a) an object orientedframework and or library; (b) source code for expressing the main codeand data used to carry out the logic of the application, whether to beexpressed as one executable image or an integrated set of executableimages; (c) digitally expressible resources; and (d) system calls orinstruction invocations necessary for connecting the logic of theapplication to the native underlying hardware and software of thedevice(s).

In another embodiment (2) , the invention provides the method of (1) ,where the system calls or instructions are used to connect the logic ofthe application to one or more of the following native underlyinghardware and software of the device: (a) software engine; (b) softwareoperating system; (c) communications subsystem; (d) graphics subsystem;(e) cryptographic subsystem; (f) security subsystem; (g) storage, audiorendering, and or input/output subsystems; (h) native code expressedalgorithms or procedures; (i) media compression and or decompressionsubsystems; (j) data processing or database processing; (j) devicespecific unique functions and capabilities exposed through applicationprogram interfaces; (k) device specific profile information forretrieving information about the resources, content and capabilities ofthe device; (l) an event queue and associated event queue managementsubsystem to drive the synchronous and asynchronous operations of theapplication across devices; (m) user interface, text, audio, video orother transcoding or rendering operations, and or general computationoperations, and or general database operations; (n) power management,suspend/resume, and or application level error recovery fromintermittent connections through the use of events to drive the sharingand cooperative operations of software, data, content and state withinand or between one or more cooperating devices; (o) subsystems todynamically save, configure, optimize and or send parts, packages ofparts, procedures, executable images, or packages of executable imagesto or from physical storage, or to and from other connected devices; and(p) any combination of these.

In another embodiment (3) , the invention provides the method of (1) ,wherein the framework or library and/or source code includes classdefinitions and object implementations to carry out, encapsulate, orderaccess to, and/or organize one or more of the native underlying softwareor hardware accessible which are listed in (a)-(p).

In another embodiment (4) , the invention provides the method of (1) ,wherein the framework or library and/or source code includes classdefinitions and object implementations to provide a basis for conformingto specific runtime conventions used for event driven execution on oramongst one or more cooperating devices.

In another embodiment (5) , the invention provides the method of (1) ,wherein the framework or library and/or source code includes classdefinitions and object implementations to ensure synchronous processingof events within an executable, and synchronized operation betweendevices by serializing the processing of events between parts of thesoftware application distributed and then run on a set of cooperatingdevices.

In another embodiment (6) , the invention provides the method of (1) ,wherein the framework comprises the DartFramework.

In another embodiment (7) , the invention provides the method of (1) ,wherein source code is expressed at least in part in any one or more of:a version or extension of the C programming language, a version orextension of the C++ programming language, a processor assemblylanguage, an interoperability instruction set, the DartInstructionSet,or any other suitable high-level, mid-level, low-level or machinelanguage.

In another embodiment (8) , the invention provides the method of (1) ,wherein the interoperability software programming language is anextension of an existing program language for expressing a singleapplication executable image with one or more of the followingextensions: (i) semantics for specifying resources to be made into partsof the output package; (ii) semantics for specifying independentlyexecutable procedures to be made into parts of the output package thatis the result of processing the source; (iii) semantics for referencingparts or ids of the parts; (iv) semantics for specifying separatelyexecutable image starting points; (v) semantics for referencing theseparately executable images to be encapsulated into the output packageor packages that is the result of processing of the source; and (vi) anycombination of these

In another embodiment (9) , the invention provides the method of (8) ,wherein the existing application program language comprises the C or C++programming language.

In another embodiment (10) , the invention provides the method of (9) ,wherein one or more of the extensions are expressed as #pragmastatements.

In another embodiment (11) , the invention provides the method of (9) ,wherein one or more of the extensions are expressed as builtin functionswith reserved names known to the software tools that parse and processesthe source code expressed in the application program language.

In another embodiment (12) , the invention provides the method of (9,wherein the extensions are parsed and processed by the DartTools.

In another embodiment (13) , the invention provides the method of (11) ,wherein the extensions are parsed and processed by the DartTools.

In another embodiment (14) , the invention provides the method of (1) ,wherein the digitally expressible resources are selected from the set ofresources consisting of one or more of: (i) a JPEG, PNG, GIF, TIFF,MPEG, or any other picture, video, audio or other media formatsresource; (ii) a Darts or any other executable package format resource,whether intended to run on one or a plurality of devices; (iii) aContent resource; (iv) a Code resource; (v) a Dataset resource; (vi) aDatabase resource; (vii) an Index resource; (viii) any other digitallyrepresentable unit or package of information resource; and (ix) anycombination of theses resources.

In another embodiment (15) , the invention provides the method of (1) ,wherein the interoperability software application is expressed as aDart.

In another embodiment (16) , the invention provides the method of (2) ,wherein the software engine is the DartEngine.

In another embodiment (17) , the invention provides the method of (2) ,wherein the system calls or instructions used to connect the logic ofthe application are from an Interoperability Instruction Set.

In another embodiment (18) , the invention provides the method of (17) ,wherein the Interoperability Instruction Set is the DartInstructionSet.

In another embodiment (19) , the invention provides the method of (2) ,wherein the device specific unique functions and capabilities exposedthrough application program interfaces are implemented in the hardwareabstraction layer of the DartEngine through the use of theOEM_BUILTIN_INSTRUCTION

In another embodiment (20) , the invention provides the method of (2) ,wherein the device specific profile information for retrievinginformation about the resources, content and capabilities of the deviceare accessed from the hardware abstraction layer of the DartEnginethrough the use of the PROFILE_INSTRUCTION.

In another embodiment (21) , the invention provides the method of (2) ,wherein the subsystems to dynamically save, configure, optimize and orsend parts, packages of parts, procedures, executable images, orpackages of executable images to or from physical storage, or to andfrom other connected devices carry out these operations according to acreationism procedure.

In another embodiment (22) , the invention provides the method of (2) ,wherein the event queue management are carried out according to aRecruitment method.

In another embodiment (23) , the invention provides the method of (2) ,wherein the power management, suspend/resume, and or application levelerror recovery are carried out using a LinearTasking method and/or aVerticalLayering method.

In another embodiment (24) , the invention provides the method of (3) ,wherein the framework or library and or source code are theDartFramework.

In another embodiment (25) , the invention provides the method of (4) ,wherein specific runtime conventions are the conventions of theDartRuntime method.

In another embodiment (26) , the invention provides an apparatus forspecifying a software application package of digitally encoded data,code and content, along with meta information in the form of data, codeand content needed to carry out an intended purpose (intent) on one ormore connected or intermittently connected devices; the apparatuscomprising: a processor and a memory coupled with the processor; andmeans accessible to the processor for expressing in an interoperabilitysoftware programming language one or more or any combination of thefollowing: (a) an object oriented framework and or library; (b) sourcecode for expressing the main code and data used to carry out the logicof the application, whether to be expressed as one executable image oran integrated set of executable images; (c) digitally expressibleresources; and (d) system calls or instruction invocations necessary forconnecting the logic of the application to the native underlyinghardware and software of the device(s).

In another embodiment (27) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to specify an application package of digitally encoded data, codeand/or content, optionally along with meta information in the form ofdata, code and/or content needed or desirable to carry out an intendedpurpose on one or more connected or intermittently connected devices,the program module including instructions for expressing in aninteroperability software programming language one or more or anycombination of the following: (a) an object oriented framework and/orlibrary; (b) source code or executable code for expressing the main codeand data used to carry out the logic of the application, whether to beexpressed as one executable image or an integrated set of executableimages; (c) digitally expressible resources; and (d) system calls orinstruction invocations necessary for connecting the logic of theapplication to the native underlying hardware and software of thedevice(s).

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

V. DartFramework/Interoperability Framework

Recall that the DartFramework is the portion of the DartSource providedfor use by programmers in building interoperability applications whichencapsulate access to many of the advantageous features of theDartPlatform eliminating the need for the programmer to have tounderstand and implement many of the desired interoperability featuresof the DartPlatform. Additional aspects of the InteroperabilityFramework including aspects of the more particular DartFramework arealso described relative to the linear tasking section of thisdescription.

In one embodiment (1) , the invention provides a method for specifyingan object oriented interoperability framework having a set of objectoriented class definitions, and implementation code thereof to be usedas part of the source specifications of an event driven softwareapplication package, the method comprising: (i) specifying using anobject oriented language as a base event processing class that containsat least the following data and code members: (a) a process member thattakes as a parameter a reference or copy of an instance of an event datastructure; and (b) an ordered list member of references to or instancesof other event processing objects which have the same base class; and(ii) implementing the members and methods of the class specification insource code.

In another embodiment (2) , the invention provides the method of (1) ,wherein the software application package is designed and implemented tocarry out an intended purpose (intent) on one or more connected orintermittently connected devices.

In another embodiment (3) , the invention provides the method of (2) ,wherein the software application package conforms to the DartFormat.

In another embodiment (4) , the invention provides the method of (1) ,wherein the base class is the Dart Gizmo class.

In another embodiment (5) , the invention provides the method of (1) ,where there is also a rendition class that inherits from the base classwhich serves as the root of a tree of inheriting instances of the baseclass where the parameter of the process member is optionally an emptyor NULL reference.

In another embodiment (6) , the invention provides the method of (5) ,where the rendition class is used to signify the beginning entry pointfor execution of a separately executable image that is to be embodied inthe output package or packages to be generated from the source code thatmakes use of the object oriented interoperability framework.

In another embodiment (7) , the invention provides the method of (5) ,where a master rendition class is specified as part of the frameworkwherein the base class is the Dart Gizmo class which inherits from therendition class that contains a list of references to or instances ofrendition objects.

In another embodiment (8) , the invention provides the method of (7) ,where the execution of a specified method of the master rendition classresults in the filling in of its list of renditions that serve as thestarting point for independently executable images that are to beincluded in the output package or output packages that result from theprocessing of the source code with makes use of the framework.

In another embodiment (9) , the invention provides the method of (5) ,wherein the rendition class is a Dart Rendition class which inheritsfrom the Gizmo base class.

In another embodiment (10) , the invention provides the method of (7) ,wherein the master rendition class is a Dart MasterRendition class whichinherits from the Dart Rendition class.

In another embodiment (1) , the invention provides the method of (2) ,wherein there are object oriented class definitions and/orimplementation code are for performing one or more of the followingfunctions or operations: (i) rendering, and/or managing and/or editingof any one or more of video, audio, pictures, images, animations, text,symbols, graphics or any other media type, code, content or datarepresentable as a sequence of digital numbers; (ii) intelligentlylimiting or ordering access to data, resources, and/or code, to one ormore of database records, fields of records, sound, video, audio, textfrom different processing units; (ii) container classes for collecting,ordering, editing, managing and controlling access to object instancesor structures; and (iv) user interface elements and the processingthereof including any one or more of the following: menus, buttons,selection boxes, text boxes, radio buttons, checkboxes, dropdownselection boxes, tables, windows or any other user input or deviceoutput used to interact with a user or software running on an externalprocessor.

In another embodiment (12) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to specify an object oriented interoperability framework having aset of object oriented class definitions and implementation code thereofto be used as part of the source specifications of an event drivensoftware application package, the program module including instructionsfor: (i) specifying using an object oriented language as a base eventprocessing class that contains at least the following data and codemembers: (a) a process member that takes as a parameter a reference orcopy of an instance of an event data structure; and (b) an ordered listmember of references to or instances of other event processing objectswhich have the same base class; and (ii) implementing the members andmethods of the class specification in source code.

In another embodiment (13) , the invention provides a computer programproduct as in (12, wherein the application package is designed andimplemented to carry out an intended purpose (intent) on one or moreconnected or intermittently connected devices.

In another embodiment (14) , the invention provides a computer programproduct as in (12) , wherein there are object oriented class definitionsand/or implementation code for performing one or more of the followingfunctions or operations: (i) rendering, and/or managing and/or editingof any one or more of video, audio, pictures, images, animations, text,symbols, graphics or any other media type, code, content or datarepresentable as a sequence of digital numbers; (ii) intelligentlylimiting or ordering access to data, resources, and/or code, to one ormore of database records, fields of records, sound, video, audio, textfrom different processing units; (ii) container classes for collecting,ordering, editing, managing and controlling access to object instancesor structures; and (iv) user interface elements and the processingthereof including any one or more of the following: menus, buttons,selection boxes, text boxes, radio buttons, checkboxes, dropdownselection boxes, tables, windows or any other user input or deviceoutput used to interact with a user or software running on an externalprocessor.

In another embodiment (15) , the invention provides an apparatus forspecifying an object oriented interoperability framework having a set ofobject oriented class definitions, and implementation code thereof to beused as part of the source specifications of an event driven softwareapplication package, the apparatus comprising: a processor logic and amemory coupled with the processor logic; means, accessible by theprocessor, specifying using an object oriented language as a base eventprocessing class that contains at least the following data and codemembers: (a) a process member that takes as a parameter a reference orcopy of an instance of an event data structure; and (b) an ordered listmember of references to or instances of other event processing objectswhich have the same base class; and means for implementing the membersand methods of the class specification in source code.

In another embodiment (16) , the invention provides an apparatus as in(15) , wherein the means specifying using an object oriented language asa base event processing class comprises a computer program or a computerprogram product.

In another embodiment (17) , the invention provides an apparatus as in(15) , wherein the means for implementing the members and methods of theclass specification in source code comprises a computer program or acomputer program product.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

VI. DartTools/Interoperability Tools

Recall that the DartTools process the DartSource applicationspecification into the Dart application packages.

In one embodiment (1) , the invention provides a method for generatingan interoperability software application package of digitally encodedinformation, along with meta information needed to carry out an intendedpurpose (intent) on one or more connected or intermittently connecteddevices; the method comprising: processing of source materials throughan interoperability compiler process [software product] to create objectfiles; and processing the object files and optional libraries through aninteroperability linker process to create libraries or aninteroperability software application package.

In another embodiment (2) , the invention provides the method of (1) ,wherein the digitally encoded information comprises digitally encodeddata, code and/or content and the meta information comprises metainformation in the form of data, code and/or content.

In another embodiment (3) , the invention provides the method of (1) ,wherein the interoperability compiler process is implemented as acompiler computer program software product and the linker process isimplemented as a linker computer program software product.

In another embodiment (4) , the invention provides the method of (1) ,wherein the source materials are assembled according to anInteroperability Source method.

In another embodiment (5) , the invention provides the method of (4) ,wherein the Interoperability source method includes a procedure forspecifying a software application package of digitally encoded data,code and content, along with meta information in the form of data, codeand content needed to carry out an intended purpose (intent) on one ormore connected or intermittently connected devices; and the methodcomprising expressing in an interoperability software programminglanguage one or more or any combination of the following: (a) an objectoriented framework and or library; (b) source code for expressing themain code and data used to carry out the logic of the application,whether to be expressed as one executable image or an integrated set ofexecutable images; (c) digitally expressible resources; and (d) systemcalls or instruction invocations necessary for connecting the logic ofthe application to the native underlying hardware and software of thedevice(s).

In another embodiment (6) , the invention provides the method of (1) ,wherein the compiler and linker are combined into a singlecompiler/linker software tool product.

In another embodiment (7) , the invention provides the method of (1) ,wherein an optional master software application package is optionallyfurther processed into one or more other interoperability softwarepackages.

In another embodiment (8) , the invention provides the method of (7) ,wherein the optional master software application package is optionallyprocessed into other interoperability software packages by the use of aninteroperability master player software product.

In another embodiment (9) , the invention provides the method of (1) ,wherein the source materials comprise DartSource source materials.

In another embodiment (10) , the invention provides the method of (1) ,wherein the interoperability software application comprises a Dartconforming to the DartFormat.

In another embodiment (11) , the invention provides the method of (10) ,wherein the Dart conforming to the DartFormat is operable within anobject oriented interoperability framework having a set of objectoriented class definitions, and implementation code thereof to be usedas part of the source specifications of an event driven softwareapplication package, the object oriented interoperability frameworkformed according to a procedure comprising: (i) specifying using anobject oriented language as a base event processing class that containsat least the following data and code members: (a) a process member thattakes as a parameter a reference or copy of an instance of an event datastructure; and (b) an ordered list member of references to or instancesof other event processing objects which have the same base class; and(ii) implementing the members and methods of the class specification insource code.

In another embodiment (12) , the invention provides the method of (1) ,wherein the interoperability compiler comprises the DartCompiler.

In another embodiment (13) , the invention provides the method of (1) ,wherein the interoperability linker comprises the DartLinker.

In another embodiment (14) , the invention provides the method of (7) ,wherein the optional master software application comprises a DartMaster.

In another embodiment (15) , the invention provides the method of (8) ,wherein the interoperability master player comprises a DartMasterPlayer.

In another embodiment (16) , the invention provides the method of (1) ,wherein the source materials include code and data references in asoftware programming language which has been extended to include one ormore or any combination of the following semantics: (1) first semanticsfor specifying resources to be made into parts of the output package;(2) second semantics for specifying independently executable proceduresto be made into parts of the output package that is the result ofprocessing the source; (3) third semantics for referencing parts or idsof the parts of the output packages; (4) fourth semantics for generatingthe parts or ids of the parts of the output packages for use at runtimeby code or data structure instances in the source code; (5) fifthsemantics for specifying separately executable image starting points;(6) sixth semantics for referencing the separately executable images tobe encapsulated into the output package or packages that is the resultof processing of the source; and (7) seventh semantics for specifyingthe processing needed to direct the compiler and/or the linker and/orthe master player software products to include at least one of parts,corresponding part table entries, and other data in the package foridentifying pointer variables which are to effectively point to aprivate memory address space independent from all other such pointers,procedures, main program data, main and program code.

In another embodiment (17) , the invention provides the method of (16) ,wherein the application program language is a version of the C or C++programming language.

In another embodiment (18) , the invention provides the method of (16) ,wherein one or more of the extensions are expressed as C or C++#pragmastatements.

In another embodiment (19) , the invention provides the method of (16) ,wherein one or more of the extensions are expressed as builtin functionswith reserved names known to the interoperability compiler that parsesand processes the source code expressed in the application programlanguage.

In another embodiment (20) , the invention provides the method of (16) ,wherein the software programming language is the DartInstructionSet orany other programming language which is known to be Turing Complete.

In another embodiment (21) , the invention provides the method of (16) ,where the interoperability compiler and interoperability linker includeparsing and processing of program statements conforming to the semanticsof one or more of the listed semantics.

In another embodiment (22) , the invention provides the method of (16) ,wherein the resources are selected from the set consisting of one ormore or any combination of the following: (i) an external file or anyother binary data image accessible by the compiler, linker and/or masterplayer procedures or software products; (ii) a data structure instancespecified in the source code; (iii) a procedure or function whether ornot the procedure is a DartProcedure specified in the source code; (iv)an executable program, an executable package, or a Dart; and (v) anycombination of any number of the above whether stored separately,packaged together, and/or compressed and/or encrypted.

In another embodiment (23) , the invention provides the method of (16) ,wherein the resources are included by reference in a #pragma statementwhich causes the compiler, linker and/or optional master playerprocedure or software products to include the resources as parts of theoutput package.

In another embodiment (24) , the invention provides the method of (16) ,wherein the parts are linear contiguous binary images to be placed intopackages or Dart parts assigned and referenced as part of a package by ascalar identifier partId with a value which is unique in the package.

In another embodiment (25) , the invention provides the method of (16) ,wherein a part table is also generated by the compiler, linker, and/ormaster player as part of the output packages with information used tofind the part image inside the package, and to provide parameters ordescriptive information relevant to the part.

In another embodiment (26) , the invention provides the method of (25) ,wherein the parameters or descriptive information are one or more or anycombination of the following: (i) a scaler or text describing thecontent type of the part; (ii) an offset into the package where the partstarts; (iii) an offset into the package where the part ends, or alength of the part so that the part end can be computed; (iv) a flags orplurality of flags to indicate special processing options for the partaccording to the content type for the loading, usage and or saving ofthe package or part; and (v) optionally other content type specificparameters which are specific to the content type.

In another embodiment (27) , the invention provides the method of (25) ,wherein the part table is the Dart Part Table conforming to theDartFormat.

In another embodiment (28) , the invention provides the method of (16) ,wherein the pointer variables are to function as virtual pointers.

In another embodiment (29) , the invention provides the method of (16) ,wherein the pointer variables are Dart VirtualPointers.

In another embodiment (30) , the invention provides a method as in (16), wherein the second semantic executable procedures includeDartProcedures.

In another embodiment (31) , the invention provides a method as in (16), wherein the second semantics take the form conforming generally to thefollowing: #pragma Procedure(<name of function>).

In another embodiment (32) , the invention provides a method as in (16), wherein the first semantics take the form conforming generally to oneor more of the following: (1) #pragma FileToPart(<file path andname><parameters such as content type, picture size destined forparameters of Part TableRecord>); or (2) #pragma partvariable(<variableor structure name>).

In another embodiment (33) , the invention provides a method as in (16), wherein the fourth semantic ids are PartIds used in the PartTableRecords and other sections of the DartFormat.

In another embodiment (34) , the invention provides a method as in (16), wherein the third semantics take the form conforming generally to thefollowing: partnumberof(<identifier name>).

In another embodiment (35) , the invention provides a method as in (16), wherein the seventh semantics take the form conforming generally tothe following: #pragma virtualpointer (<pointer variableidentifier><values to control optional automatic saving of values when aDart SAVE_INSTRUCTION is executed and to control the number of realmemory pages to be used>).

In another embodiment (36) , the invention provides the method of (1) ,wherein the interoperability compiler, interoperability linker andinteroperability master player are the DartTools.

In another embodiment (37) , the invention provides the method of (1) ,wherein the interoperability software application package comprises aDart conforming to the DartFormat.

In another embodiment (38) , the invention provides the method of (8) ,where as an option, the source code specifies a single masterindependently executable image that is output in the package generatedby the compiler and/or the linker along with extra parts containingobject class, instance, and linkage information used when the package isexecuted on an optional software tool product or the master player, tointelligently generate one or more packages.

In another embodiment (39) , the invention provides the method of (38) ,wherein a single master independently executable image execution entrypoint is an entry point to a method of a special derived class from thebase event processing class, with the following added features: (i) dataelements and/or methods to form, hold and manage a list of specialobject base class or derived class object instances and/or references tosuch instances that are to be output in a particular package; and (ii)one or more setup methods which are called when the master is executed.

In another embodiment (40) , the invention provides the method of (38) ,wherein a single master independently executable image execution entrypoint is an entry point to a method of a special derived class from thebase event processing class, the method being for specifying an objectoriented interoperability framework having a set of object orientedclass definitions, and implementation code thereof to be used as part ofthe source specifications of an event driven software applicationpackage, and the method comprising: (i) specifying using an objectoriented language as a base event processing class that contains atleast the following data and code members: (a) a process member thattakes as a parameter a reference or copy of an instance of an event datastructure; and (b) an ordered list member of references to or instancesof other event processing objects which have the same base class; and(ii) implementing the members and methods of the class specification insource code; the method further including: (i) providing data elementsand/or methods to form, hold and manage a list of special object baseclass or derived class object instances and/or references to suchinstances that are to be output in a particular package; and (ii)providing one or more setup methods which are called when the master isexecuted.

In another embodiment (41), the invention provides the method of (39) ,wherein the execution of the master results according to thespecifications of the source in the forming of the list.

In another embodiment (42) , the invention provides the method of (38) ,wherein the special derived class is a Dart Rendition class.

In another embodiment (43) , the invention provides the method of (39) ,wherein the single master independently executable image is a DartRendition inside a Dart.

In another embodiment (44) , the invention provides the method of (41) ,wherein as an option the execution which results in the forming of thelist also include configuring, creating, optimizing, adding parts to orotherwise affecting the independently executable images that arereferenced by the list.

In another embodiment (45) , the invention provides the method of (44) ,wherein the list is formed and the configuring, creating, optimizingand/or adding parts to or otherwise affecting the generation or makeupof independently executable images is performed using input from or moreof the following sources: (i) a human input solicited during themaster's execution; (ii) an automated procedure that was specified bythe source code; (iii) an information collected from the computingdevice on which the master is executing by the executing code of themaster package; (iv) an information collected from any number ofcomputing devices over any number of communications mediums by theexecuting code of the master package; and (v) combinations of these.

In another embodiment (46) , the invention provides the method of (39) ,wherein as a step for each package to be produced by executing themaster on a master player, the generated list is used to find thestarting points for all the independently executable images and theassembly of the package is performed according to a procedure comprisingthe steps: (i) identifying the execution entry point of one of thederived class object instances and or references on the list; (ii)recursively or interactively tracing all the possible execution pathsstarting at the execution entry point while keeping track of all thereachable elements; (iii) generating all the code, data, resources, andother digitally representable parts necessary to form a singleexecutable image for placement in a package which includes all thereachable elements that have been tracked; (iv) performing steps (i),(ii) and (iii) for each of the derived class object instances and orreferences on the list; and (v) generating a package of individuallyexecutable images.

In another embodiment (47) , the invention provides the method of (39) ,wherein the generating of a package of individually executable imagescomprises at least one of the following: a) meta information necessaryfor locating, loading, and extracting each individual executable image;and b) parts which together contain all the reachable elements that havebeen tracked for all the separately executable images.

In another embodiment (48) , the invention provides the method of (39) ,wherein the recursively or interactively tracing all the possibleexecution paths starting at the execution entry point while keepingtrack of all the reachable elements, includes keeping track of allreachable variables, data structure instances, functions and all theobjects that are pointed to or otherwise referenced by the data in or bythe methods, variables, data structure instances, functions and objects,and further iteratively or recursively all such elements to which thesepoint to or reference.

In another embodiment (49) , the invention provides the method of (46) ,wherein elements not tracked as being reachable are not generated asmeta information or parts or any other representation of the package toreduce the size and or complexity of the package, or the handling orexecution of the package or parts of the package.

In another embodiment (50) , the invention provides the method of (38) ,wherein processing of the master by a master player is used for one ormore or any combination of the following purposes: (i) as a programgenerator for dynamic customization of one or a plurality of packages;(ii) to collect real-time information that is to be part of thegenerated package or packages that was not collected or was notobtainable at the time of compiling and or linking of the master; (iii)as a graphical user interface for interactively selecting, laying out,and/or providing new information needed to generate particular packages;(iv) for creating, and/or viewing, and/or testing, and/or correcting,and/or building various aspects of various independently executableimages or sets of independently executable images to be output in one ormore packages in an optionally interactive manner with optional visualfeedback of the changing characteristics of the package or packages tobe output; and (v) for dynamically generating tables, code, content, orother resources, or collections thereof to be used as parts of thegenerated package or packages.

In another embodiment (51) , the invention provides the method of (49) ,wherein the processing of the master beneficially avoids the need torecompile and or relink or otherwise re-process the source code.

In another embodiment (52) , the invention provides the method of (49) ,wherein the user interface is presented as part of the execution of themaster for customizing the package generation.

In another embodiment (53) , the invention provides the method of (52) ,wherein the user interface is presented according to the code, dataand/or resources of the master.

In another embodiment (54) , the invention provides the method of (48) ,wherein some subset of the code, data and or resources that are elementsof the master for carrying out the purposes are referenced only as theresult of execution starting at execution entry points of derived classobject instances that are not on the list.

In another embodiment (55) , the invention provides the method of (49) ,wherein some subset of the code, data and or resources that are elementsof the master for carrying out the purposes are referenced only as theresult of execution starting at execution entry points of derived classobject instances that are not on the list.

In another embodiment (56) , the invention provides the method of (54) ,wherein the subset is left out of the generated package.

In another embodiment (57) , the invention provides the method of (56) ,wherein the subset is left out to decrease the size of the package orreduce the complexity of the package.

In another embodiment (58) , the invention provides the method of (46) ,wherein tracking information about reachable elements or sets ofelements also includes further information for each such element or setof elements an access list containing on this access list all theentries from which the particular element or set of elements isreachable.

In another embodiment (59) , the invention provides a method as in (46), wherein the individual part binary images are sharable between theindependently executable images so that the size of an interoperabilityapplication may be limited.

In another embodiment (60) , the invention provides the method of (59) ,wherein the further information is used by the master player product toproduce a package or packages where parts image instances inside thegenerated package are shared amongst the separately executable imagesrather than having to duplicate such part images for each separatelyexecutable image in the package which logically contains the part image.

In another embodiment (61) , the invention provides the method of (58) ,wherein one or more procedural elements of the package arecompiled/linked and optionally processed by the master player into aform conforming to a Interoperability Instructions Set.

In another embodiment (62) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to generate an interoperability software application package ofdigitally encoded information, optionally along with meta informationneeded to carry out an intended purpose on one or more connected orintermittently connected devices, the program module includinginstructions for: processing of source materials through aninteroperability compiler process [software product] to create objectfiles; and processing the object files and optional libraries through aninteroperability linker process to create libraries or aninteroperability software application package.

In another embodiment (63) , the invention provides a computer programproduct as in (62) , wherein the digitally encoded information comprisesdigitally encoded data, code and/or content and the meta informationcomprises meta information in the form of data, code and/or content.

In another embodiment (64) , the invention provides a computer programproduct as in (62) , wherein the interoperability compiler process isimplemented as a compiler computer program software product and thelinker process is implemented as a linker computer program softwareproduct.

In another embodiment (65) , the invention provides a computer programproduct as in (62) , wherein the source materials are assembledaccording to an Interoperability Source method.

In another embodiment (66) , the invention provides a computer programproduct as in (65) , wherein the Interoperability source method includesa procedure for specifying a software application package of digitallyencoded data, code and content, along with meta information in the formof data, code and content needed to carry out an intended purpose(intent) on one or more connected or intermittently connected devices;and the method comprising expressing in an interoperability softwareprogramming language one or more or any combination of the following:(a) an object oriented framework and or library; (b) source code forexpressing the main code and data used to carry out the logic of theapplication, whether to be expressed as one executable image or anintegrated set of executable images; (c) digitally expressibleresources; and (d) system calls or instruction invocations necessary forconnecting the logic of the application to the native underlyinghardware and software of the device(s).

In another embodiment (67) , the invention provides an interoperabilitysoftware products tool set whether packaged separately, dynamicallylinked together, or packaged into a single executable in any combinationcomprising: a) an interoperability compiler; b) an interoperabilitylinker; and c) an interoperability master player.

In another embodiment (68) , the invention provides the interoperabilitysoftware products tool set of (67) , wherein the interoperabilitycompiler and linker in combination execute a method for generating aninteroperability software application package of digitally encodedinformation, along with meta information needed to carry out an intendedpurpose (intent) on one or more connected or intermittently connecteddevices; the method comprising: processing of source materials throughan interoperability compiler process to create object files; andprocessing the object files and optional libraries through aninteroperability linker software product to create libraries or aninteroperability software application package.

In another embodiment (69) , the invention provides the interoperabilitysoftware products tool set of (67) , wherein the master softwareapplication package is optionally processed into other interoperabilitysoftware packages by the use of an interoperability master playersoftware product.

In another embodiment (70) , the invention provides the interoperabilitysoftware products tool set of (67) , wherein the software product orproducts are implemented and or designated and or used as DartTools.

In another embodiment (71) , the invention provides the interoperabilitysoftware products tool set of (67) , wherein there is a single selectionprocedure or set of selection procedures associated with each renditionto be used in a recruitment procedure to intelligently select the mostsuitable independently executable image, from the available images, torun on a target device and or target device environment.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

VII. DartFormat/Interoperability Format

Recall that the DartFormat comprise the structure and the rules forputting together a Dart format package which encapsulates all the code,data, and content needed for an interoperability applications which canthen be loaded and run on DartDevices, which contain a runningDartPlayer.

In one embodiment (1) , the invention provides a method for storing asoftware application package conforming to an interoperability format ofdigitally encoded data, code and content, along with meta information inthe form of data, code and content needed to carry out an intendedpurpose (intent) on one or more connected or intermittently connecteddevices, the method comprising: forming at least one linearly contiguousbinary encoded part image; forming any necessary linearly contiguouspart images comprised of combinations of binary encoded resources orprogram elements that are to be used to identify, load, select,identify, execute or be processed as part of the application package;forming meta information; and packaging the parts and meta informationtogether in a form where part images can be deterministically locatedand independently executable images identified, loaded, and executed.

In another embodiment (2) , the invention provides a method as in (1) ,wherein the at least one linearly continuous binary encoded part imagecontains at least one of a main code, a main data, a table of recordsfor each of the independently executable images, and a table of recordsfor each of the parts belonging to a particular independently executableimage.

In another embodiment (3) , the invention provides a method as in (1) ,wherein the at least one linearly continuous binary encoded part imagecontains each of a main code, a main data, a table of records for eachof the independently executable images, and a table of records for eachof the parts belonging to a particular independently executable image.

In another embodiment (4) , the invention provides a method as in (1) ,wherein the necessary linearly contiguous part images optionallyincluding any number or combination selected from the set consisting of:(a) programs, Darts, DartProcedures, or procedures; (b) pictures, video,image, audio, sound or any other media capable of being rendered; (c)data structure instances, lists, and or parameters; (d) lists of data ortables of data; and (e) any combination of these.

In another embodiment (5) , the invention provides a method as in (1) ,wherein the forming meta information includes forming meta informationof at least one of: (a) a table for finding parts given its identifier;(b) a first information used to find the table of parts; (c) a secondinformation needed to load and execute the linearly contiguous partimages; and (d) a third information used to find the list ofindependently executable images.

In another embodiment (6) , the invention provides the method of (1) ,wherein one or more of the following linearly contiguous binary encodedpart images are also formed: (i) procedures, Darts, DartProcedures; (ii)content including content selected from the set of content itemsconsisting of pictures, audio, video, or other multimedia content oranimations; (iii) databases; (iv) indexes; (v) parameters for list itemsor table items; (vi) virtual pointer data; (vii) application heap data;and (viii) anything else expressible as a contiguous binary data image.

In another embodiment (7) , the invention provides the method of (1) ,wherein one or more of the following meta information are also formed:(i) signature information; (ii) keywords or other information to beaccessed to identify the names, types, content and or uses of thesoftware application package; (iii) parameters for list items or tableitems; (iv) virtual pointer parameters; (v) security checksums, and orsignatures, and or certificates and or hashes; and (vi) uniqueidentifiers.

In another embodiment (8) , the invention provides the method of (1) ,wherein the steps are performed by tools embodied in theinteroperability compiler, interoperability linker and orinteroperability master player.

In another embodiment (9) , the invention provides the method of (1) ,wherein any one, any combination, or all of the following are true: (i)the format for the storing a software application package is theDartFormat; (ii) the software application package is a Dart; (iii) theparts are DartParts; (iv) the table of independently executable imagesis a Dart RenditionsTable; (v) the table of records for each of theparts is a Dart RenditionTable; (vi) the table for finding parts givenits identifier is a Dart Part Table; and (vii) the Information used tofind the table of parts is the DartTrailer and its known size, fields,and location in the DartFormat image.

In another embodiment (10) , the invention provides the method of (1) ,wherein the main code is comprised of instructions from aninteroperability instruction set.

In another embodiment (11) , the invention provides the method of (1) ,wherein interoperability instruction set is the Dart InteroperabilityInstruction set.

In another embodiment (12) , the invention provides the method of (2) ,wherein the main code contains all the code not otherwise explicitlydesignated in the originating source to go into an independent part.

In another embodiment (13) , the invention provides the method of (12) ,wherein the main code instruction data address fields reference dataassigned to one linear data address space in the main data part.

In another embodiment (14) , the invention provides the method of (12) ,wherein the main code instruction branching fields reference other codeassigned to one linear address space in the main code part.

In another embodiment (15) , the invention provides the method of (2) ,wherein the main data contains all the data instances not otherwiseexplicitly designated in the originating source to go into anindependent part.

In another embodiment (16) , the invention provides the method of (2) ,wherein the table of independently executable images contains tablerecords with at least the part identifier of a table for finding partsgiven its identifier

In another embodiment (17) , the invention provides the method of (2) ,wherein the records of the table of independently executable imagescontains one or more of the following optional fields in anycombination: (i) the smallest linear address range of the main code partthat contains all the code needed for the independently executableimage; (ii) the smallest linear address range of the main data part thatcontains all the data needed for the independently executable image; and(iii) the requested or required amount of memory or other resources thatis to be reserved to be used during execution of the independentlyexecutable image.

In another embodiment (18) , the invention provides the method of (6) ,wherein the Virtual pointer data includes any number of address rangesfollowed by the data instance values of the address range so as toefficiently represent a possibly sparse list of ranges of data values.

In another embodiment (19) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for storing a software application package conforming to aninteroperability format of digitally encoded data, code and content,along with meta information in the form of data, code and content neededto carry out an intended purpose (intent) on one or more connected orintermittently connected devices, the program module includinginstructions for: forming at least one linearly contiguous binaryencoded part image; forming any necessary linearly contiguous partimages comprised of combinations of binary encoded resources or programelements that are to be used to identify, load, select, identify,execute or be processed as part of the application package; forming metainformation; and packaging the parts and meta information together in aform where part images can be deterministically located andindependently executable images identified, loaded, and executed.

In another embodiment (20) , the invention provides a data structure forstoring a software application package conforming to an interoperabilityformat of digitally encoded data, code and content, along with metainformation in the form of data, code and content needed to carry out anintended purpose (intent) on one or more connected or intermittentlyconnected devices, the data structure comprising: at least one linearlycontiguous binary encoded part image; any necessary linearly contiguouspart images comprised of combinations of binary encoded resources orprogram elements that are to be used to identify, load, select,identify, execute or be processed as part of the application package;meta information; and the data structure packaging the parts and metainformation together in a form where part images can bedeterministically located and independently executable imagesidentified, loaded, and executed.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

VII. DartRuntime/Interoperability Runtime

In another aspect, the invention provides a software run-time model(such as the DartRuntime model) which is largely event driven, and insome embodiments entirely event driven, for all software operationswhether Dart or device control related. A set of event semantics, types,structures and operations on events common to both the applications andthe low-level device control software is provided by one event queuerunning on each interoperating device in a team of interoperatingdevices (FIG. 17 660) which drives the sequencing of synchronousapplication event processing and asynchronous application, device,communications and interoperability operations. Also provided are aninstruction set or system calls which are used to manage the queuing,dequeuing and processing of events. It also provides an optional robustdevice interoperability communications runtime model wherecommunications between devices is maintained, error corrected, and whennecessary reestablished with cooperation, but with only a small amountof or no disruption to Darts running effectively across a team ofinteroperating devices (see FIG. 19). These features may optionally beextended to separately generated Darts and/or other devices.

Other aspects of the DartRuntime are described elsewhere in thisspecification including in the examples and in the exemplary embodimentsbelow.

In one embodiment (1) , the invention provides a method for aninteroperability runtime system to carry out the execution of an eventdriven software application package of digitally encoded data, code andcontent, along with meta information in the form of data, code andcontent needed to carry out an intended purpose (intent) on one or moreconnected or intermittently connected devices, the method comprising:(a) selecting and loading a separately executable image from a givenpackage of independently executable images; (b) recruiting devices intoa team by the executing image; (c) distributing at least one of code,renditions, data, and/or content amongst the team of recruited devices;(d) processing in an orderly manner of synchronous and asynchronousevents to carry out the intent; (e) synchronization and serializingevent processing within and between devices of the recruited team ofdevices; and (f) saving of running packages of individually executableimages including saving data and state in a storage.

In another embodiment (2) , the invention provides the method of (1) ,wherein the (a) selecting and loading of a separately executable imagefrom a given package of independently executable images is the selectingand loading of exactly one separately executable image from a givenpackage of independently executable images.

In another embodiment (3) , the invention provides the method of (1) ,wherein the runtime is the DartRuntime.

In another embodiment (4) , the invention provides the method of (1) ,wherein the selecting and loading, recruiting, distributing, processing,and synchronizing and reserializing are carried out according to the arecruitment procedure.

In another embodiment (5) , the invention provides the method of (1) ,wherein the saving of running packages including data and state iscarried out by the SAVE_INSTRUCTION of the DartInstructionSet.

In another embodiment (6) , the invention provides the method of (1) ,wherein the saving of running packages including data and statecomprises: getting a pointer to a table which contains a plurality ofrecords and where each record provides information about a singleseparately executable image; processing each record of the table inturn; and creating an interoperability software package of one or moreindividually executable images based on the processed records.

In another embodiment (7) , the invention provides the method of (6) ,wherein the table is a renditions table.

In another embodiment (8) , the invention provides the method of (6) ,further comprising optionally forming a header with one or more headerfields.

In another embodiment (9) , the invention provides the method of (6) ,further comprising optionally forming a trailer with one or more trailerfields.

In another embodiment (10) , the invention provides the method of (7) ,wherein the processing of each record of the renditions table in turncomprises: (a) getting a pointer to a rendition table which containsrecords, each renditions table record comprising: (i) a part ID (partid)which identifies or references a part image that is to be included inthe saved package; (ii) a linear contiguous address range for the codefrom the running package's main code needed to be included for theseparately executable image corresponding to the information of therecord of the renditions table being processed; and (iii) a linearcontiguous address range for the data from the running package's maindata needed to be included for the separately executable imagecorresponding to the information of the record of the renditions tablebeing processed; (b) accumulating all the partids; (c) accumulating theoutside bounds for the main code linear address ranges of all previouslyprocessed records; and (d) accumulating the outside bounds for the datacode linear address ranges of all previously processed records.

In another embodiment (11) , the invention provides the method of (7) ,wherein at least one of the created individually executable imagesincludes: (i) the renditions table; (ii) the rendition tables; (iii) amain code part with at least the range of data accumulated during theprocessing of each record; (iv) a main data part with at least the rangeof data accumulated during the processing of each record; (v) all theaccumulated parts; and (vi) a part table with a record for each part inthe package.

In another embodiment (12) , the invention provides the method of (7) ,wherein the part table with a record for each part in the packageincludes at least the following: (i) a partId that this record refersto; (ii) a starting offset within the image of the part image; and (iii)an ending offset and or length of the image of the part image.

In another embodiment (13) , the invention provides the method of (7) ,wherein the individually executable images further includes an optionalheader if one was formed.

In another embodiment (14) , the invention provides the method of (7) ,wherein the individually executable images further includes an optionaltrailer if one was formed.

In another embodiment (15) , the invention provides the method of (6) ,wherein a header and or trailer or any other data structure with a knownsame fixed offset is required in all instances of software applicationpackages.

In another embodiment (16) , the invention provides the method of (15) ,wherein the header and/or trailer and or other data structure oncelocated using the fixed offset, provides the offset of the part table orpart database data.

In another embodiment (17) , the invention provides the method of (15) ,wherein where the header and/or trailer or other data structure includesthe part table or part database data.

In another embodiment (18) , the invention provides the method of (1) ,wherein the saving of running packages including data and state iscarried out by the SAVE_INSTRUCTION of the DartInstructionSet.

In another embodiment (19) , the invention provides the method of (1) ,wherein the runtime is embodied and carried out through aninteroperability platform comprising at least the following: (1) anobject oriented framework for specifying code, data, content andresources for use in interoperability application packages; (2) a sourcefor specifying the code, data, content and resources, including that ofthe object oriented framework, for an interoperability applicationpackage; (3) a known process for storing the interoperabilityapplication package as a set of parts and meta information for finding,accessing, and loading the package or portions of the package on adevice to carry out the execution of the package and therefore carry outthe intent embodied in the application package; (4) an InteroperabilityInstruction set for providing a binary code compatibility between two ormore devices; (5) a first software product or product(s) tool(s) whichtake the source and generate one or more interoperability applicationpackage), where some or all of the code elements are represented bysequences of instructions from the interoperability instruction set; (6)a second software product which loads and executes interoperabilityapplication packages; (7) a procedure for ordering the execution of thevarious processing units embodied in the framework; (8) a procedure forintelligently spreading code, data, and content to one or more otherdevices as needed to carry out a software application; and (9) aprocedure for serializing and synchronizing the activities of the codeand data once distributed across one or more devices.

In another embodiment (20) , the invention provides the method of (19) ,further comprising providing at least one of: (i) a procedure forapplication level error recovery; (ii) a procedure for application levelpower management; (iii) a procedure for tightly coupling execution unitsby having a single semantic entity generated and used by all theexecution units whether performing high level application tasks or lowlevel communications or hardware access tasks; and (iv) a procedure forordering and managing event driven execution and runtime environments ofa plurality of event processing units of a software application.

In another embodiment (21) , the invention provides the method of (19) ,wherein one or more or any combination of the following are true: (1)the interoperability platform is as described herein elsewhere in thedescription; (2) the framework is as described herein elsewhere in thedescription; (3) the interoperability application packages are asdescribed herein elsewhere in the description; (4) the source is asdescribed herein elsewhere in the description; (5) the framework is asdescribed herein elsewhere in the description; (6) the known process forstoring the interoperability application package is as described hereinelsewhere in the description; (7) the Interoperability Instruction Setis as described herein elsewhere in the description; (8) the softwareproduct or product(s) tools are as described herein elsewhere in thedescription; (9) the software product which loads and executesinteroperability application packages is as described herein elsewherein the description; (10) the procedure for ordering the execution of thevarious processing units embodied in the framework is as described inthe Linear Tasking section of this description; and (11) the procedurefor ordering the execution of the various processing units embodied inthe framework is as described in the Linear Tasking section of thisdescription.

In another embodiment (22) , the invention provides the method of (19) ,wherein one or more or any combination of the following are true: (1)the interoperability platform is a DartPlatform; (2) the framework is aDartFramework; (3) the interoperability application packages are Darts;(4) the source is a DartSource; (5) the framework is a DartFramework;(6) the known process for storing the interoperability applicationpackage is any method for creating a package conforming to a DartFormat;(7) the Interoperability Instruction Set is a DartInstructionSet; (8)the software product or product(s) tools are a DartTools; (9) thesoftware product which loads and executes interoperability applicationpackages is a DartEngine; (10) the procedure for ordering the executionof the various processing units embodied in the framework is a DartLinear Tasking; and (11) the procedure for ordering the execution of thevarious processing units embodied in the framework is a Dart LinearTasking.

In another embodiment (23) , the invention provides the method of (19) ,wherein the intelligently spreading code, data and content to one ormore other devices as needed comprises recruitment and/or renditioning.

In another embodiment (24) , the invention provides the method of (23) ,wherein the recruitment comprises DartRecruitment and the renditioningcomprises DartRenditioning.

In another embodiment (25) , the invention provides the method of (19) ,wherein the serializing and synchronizing the activities of the code anddata once distributed across one or more devices is performed byrecruitment.

In another embodiment (26) , the invention provides the method of (25) ,wherein the recruitment comprises DartRecruitment.

In another embodiment (27) , the invention provides the method of (20) ,wherein the method for application level error recovery is as describedelsewhere in this specification.

In another embodiment (28) , the invention provides the method of (20) ,wherein the method for application level power management is asdescribed elsewhere in this specification.

In another embodiment (29) , the invention provides the method of (20) ,wherein the method for tightly coupling execution units is as describedin the Vertical Layering description elsewhere in this specification.

In another embodiment (30) , the invention provides the method of (20) ,wherein the method for ordering and managing event driven execution isas described in the recruitment runtime description elsewhere in thisspecification or is DartRecruitment.

In another embodiment (31) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner to provide an interoperability runtime system to carry out theexecution of an event driven application package of digitally encodeddata, code and/or content, along with meta information in the form ofdata, code and/or content needed to carry out an intended purpose on oneor more connected or intermittently connected devices, the programmodule including instructions for: (a) selecting and loading aseparately executable image from a given package of independentlyexecutable images; (b) recruiting devices into a team by the executingimage; (c) distributing at least one of code, renditions, data, and/orcontent amongst the team of recruited devices; (d) processing in anorderly manner of synchronous and asynchronous events to carry out theintent; (e) synchronization and serializing event processing within andbetween devices of the recruited team of devices; and (f) saving ofrunning packages of individually executable images including saving dataand state in a storage.

In another embodiment (32) , the invention provides an interoperabilityruntime system for carrying out the execution of an event drivensoftware application package of digitally encoded data, code and/orcontent, optionally along with meta information in the form of data,code and/or content needed to carry out an intended purpose (intent) onone or more connected or intermittently connected devices, the systemcomprising: (a) a processor and a memory coupled with the processor; (b)means, accessible by the processor, for selecting and loading aseparately executable image from a given package of independentlyexecutable images; (c) means, accessible by the processor, forrecruiting devices into a team by the executing image; (d) means,accessible by the processor, for distributing at least one of code,renditions, data, and/or content amongst the team of recruited devices;(e) means, accessible by the processor, for processing in an orderlymanner of synchronous and asynchronous events to carry out the intent;(f) means, accessible by the processor, for synchronization andserializing event processing within and between devices of the recruitedteam of devices; and (g) means, accessible by the processor, for savingof running packages of individually executable images including savingdata and state in a storage.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

IX. Linear Tasking

In another aspect of the invention, system, apparatus, method andcomputer program for linear tasking are provided. Linear Taskingincludes a methodology that enables a simple deterministic flow ofprocessor control and device resources between processing units.Processing units can easily be reconfigured into a single hierarchywhich includes processing units compiled into an application, separatelycompiled applications, and even applications running on other devices.

Conventional software eco-systems, especially conventional operatingsystems, are built using either a cooperative tasking model or apre-emptive tasking model. While there are advantages to theseconventional tasking models for some devices which run applicationswhich require computer response times in the milliseconds or evenmicroseconds, these advantages are outweighed by the benefits of aLinear Tasking model for the most common types of interoperabilityapplications where most often response time requirements are often under30 milliseconds. Many of the applications and tasks will only requireresponse times of tenths of seconds, and even to response times on theorder of a second to several seconds.

The advantages of a Linear Tasking model are especially well suited tointeroperability applications because most such applications areintended for individual users. Here the run-time penalty for having topass control to each processing unit is not important, as human responsetime needs are generally limited to a thirtieth of a second or longer.

Although there are many advantages of the inventive linear tasking modeland method, only a few are set forth here while others will be apparentfrom the further description in this specification.

A first advantage is a deterministic or near-deterministic path for theorder of processing of all the processing units of all the teameddevices leads to a simpler programming model and greatly reduced bugsdue to limiting the order in which operations are performed to a wellcontrolled pattern.

Second, all processing units are able to complete complex sequences ofoperations without the possibility of being interrupted in time or beingeffected by the actions of interrupting processes.

Third, there is easy support for controlled shutdowns, controlledhibernation and recovery and advantageous Vertical Layering mechanisms,that may for example be used for such applications as power management.

Fourth, arranging and rearranging the hierarchy is at least one easy wayfor enabling and disabling groups of processing units as the modes ofoperation or interaction with the user changes Fifth, environments canbe inherited, changed or distorted by parent processing units in amanner where the children do not need to contain code which knows howtheir parents are using their outputs, such as bitmaps, or controllingtheir environment, such as their sense of time.

Sixth, separately produced executables can be easily assimilated intothe processing hierarchy of a running executable. In the DartPlatform,this feature allows Darts to assimilate other Darts into its runtimeduring use. For example, a slide show (SlideShow) Dart applicationdescribed earlier can readily be built which can collect and containseparately produced Darts as slides, just as easily as it can collectstill JPEG pictures as slides. The JPEG picture would be contained by astill picture display Gizmo (Gizmo is the base calls for eventprocessing units) while a Dart would be contained by a Dart containerGizmo.

In one embodiment, Linear Tasking is advantageously embodied throughoutthe implementation of the Dart platform (DartPlatform FIG. 3) from theDart framework (DartFramework, FIG. 11, to the Dart runtime (DartRuntimeFIG. 9, FIG. 16, FIG. 17), to the Dart engine (DartEngine FIG. 22 600)and instructions in the Dart instruction set (DartInstructionSet FIG.20, FIG. 25). Other embodiments may use Linear Tasking in conjunctionwith alternative implementations with the benefit of complete Dartinfrastructure.

In one embodiment, the inventive linear tasking provides a method fordeterministically ordering the execution and runtime environments of aplurality of processing units of software application programs. Asoftware or firmware program event driven run-time model provides orcreates an environment where all processing units are executed in apredefined order based on the linkage between the processing units. Asoftware object oriented framework, the DartFramework (FIG. 11), isprovided wherein there is a base processing unit class (Gizmo in FIG. 11115) containing at least one of and possibly all of the following objectmembers (FIG. 11 115) in any combination:

-   -   (a) a possibly null reference to a parent processing unit;    -   (b) a possibly empty ordered linear list of references to a        child processing units;    -   (c) optional binary flags corresponding to classes of event        types that are to be acted upon by this processing unit;    -   (c) optional binary flags corresponding to the classes of event        types that are to be acted upon by any of the child processing        units descending down the chain of parent and child processing        until there are no more children;    -   (d) procedural methods for adding, deleting, or reordering the        references to the parent and child processing units; and    -   (e) a procedural method for ordering the processing of events.

In at least one embodiment, the procedural method for ordering theprocessing of events includes the following steps (see FIG. 15):

-   -   (1) Perform any pre-children processing of the event needed        based on the event type, its values and references the tasks of        the processing unit and the current run-time environment as        specified by the event, the parent and the state of the device        (see FIG. 15 8004);    -   (2) Set the optional flags corresponding to the classes of event        types that are to be acted upon by the child processing unit        chain to indicate that no event types are to be acted upon;    -   (3) Set up any environment changes needed for each child's        processing, and call each child in the order of the list of        references to child processing units (see FIG. 15 8005);    -   (4) As each call returns logically OR the binary flags of        processing types needed to be processed by each of the just        called child to collect the combined event processing type based        needs of all the children and their decedents;    -   (5) Perform any post-children processing of the event based on        the event type its values and references the tasks of the        processing unit and the current run-time environment as        specified by the event, the parent and the state of the device        (see FIG. 15 8004);    -   (6) Set the flags of event types that are now to be handled by        this processing unit in the future;    -   (7) Return control to the parent processing unit if the        reference is non-null (see FIG. 15 8006); and    -   (8) Return control to the main processing loop if the parent is        null FIG. 15 8008.

The collected binary flags can optionally be used for pruning of thegraph of calls since the flags can be used to indicate which event typesare not needed for processing by each child and their all itsdescendents. If the child's or'ed flag value corresponding to the eventtype classification the flag is associated with is not 1 then the childand its descendents have no need to process the event.

Other exemplary embodiments of linear tasking are now described. In oneembodiment (1) , the invention provides a method of linear tasking forordering and managing event driven execution and runtime environments ofa plurality of event processing units of a software application package,the method comprising: providing a software object oriented frameworkwhich includes a base event processing unit class, and zero or moreevent processing unit classes which inherit either directly orindirectly from the base event processing unit class; creating,maintaining, adding, deleting or reordering links that form a graph ortopology of event processing units in a manner that ensures that thereis always a single linear deterministic ordering for passing eventsthrough the graph of processing units formed by the links; anddynamically changing the graph or topology of processing units accordingto the needs of the running application.

In another embodiment (2) , the invention provides the method of lineartasking (1), wherein the graph is logically extended to include thegraphs of separately generated application packages statically byreference within one or more application packages or applicationspackages that are themselves part of one or more other applicationpackages.

In another embodiment (3) , the invention provides the method of (1) ,wherein the graph is logically extended to include the graphs ofseparately generated application packages dynamically during the runningof the application package.

In another embodiment (4) , the invention provides the method of (2) ,wherein the event processing unit at a topmost node of the separatelygenerated application package's graph determines by way of theparameters passed into a event processing method if it is logically apart of an extended graph or actually the start of a topmost applicationpackage that is not extended by any other application package.

In another embodiment (5) , the invention provides the method of (3) ,wherein the event processing unit at a topmost node of the separatelygenerated application package's graph determines by way of theparameters passed into a event processing method if it is logically apart of an extended graph or actually the start of a topmost applicationpackage that is not extended by any other application package.

In another embodiment (6) , the invention provides the method of (4) ,wherein if and only if the parameters do not specify an event toprocess, the topmost node will explicitly attempt to retrieve an eventfrom a queue of events for processing by itself and any other eventprocessing units in the linear order of the graph.

In another embodiment (7) , the invention provides the method of (5) ,wherein if and only if the parameters do not specify an event toprocess, the topmost node will explicitly attempt to retrieve an eventfrom a queue of events for processing by itself and any other eventprocessing units in the linear order of the graph.

In another embodiment (8) , the invention provides the method of (3) ,wherein a container event processing unit class is used to instantiateobjects to load, link, unload, and manage the environment of separatelygenerated application packages so that their event processing unitsbecome a logical extension of the graph.

In another embodiment (9) , the invention provides a method as in (1) ,wherein the event for ordering the processing of events is performedaccording to the following steps: (1) optionally processing unit baseclass comprises at least one of the following class object members: (1)a possibly null reference to a parent processing unit; (2) a possiblyempty ordered linear list of references to child processing units; (3)one or more optional binary flag(s) corresponding to classifications ofevent types that are to be acted upon by the processing unit itself),wherein the flags are used to eliminate unnecessary passing of eventsdown to children or their descendants that will not process any eventsof a particular class; (4) one or more child optional binary flag(s)corresponding to the classes of event types that are to be acted upon byany of the child processing units descending down a chain of parentprocessing and child processing until there are no more children orchild processing units; (5) a procedure for adding, deleting, modifying,and/or reordering a reference to at least one of the parent and childprocessing units; and (6) a procedure for ordering the processing ofevents.

In another embodiment (10) , the invention provides a method as in (9) ,wherein the event processing unit class comprises at least oneprocessing unit class object member for ordering the processing ofevents.

In another embodiment (11) the invention provides the method in (10) ,wherein the procedure for ordering the processing of events is performedaccording to the following steps: (1) optionally perform anypre-children processing of the event needed; (2) initially set (orreset) optional child binary flags maintained to track the combinedevent processing needs of all the children event processing units needsalong with the needs of all their dependents (children)), where eachoptional child flag is a single bit corresponding to exactly oneclassification of event types, to initially indicate that no events ofany of the classifications are to be acted upon by children or theirdecedents; (3) set up any environment changes needed for each childprocessing and call each child event processing unit in the order of thelist of children; (4) as each call to a child returns, logically OR theoptional child binary flags of processing types needed to be processedby each of the just called child to collect the combined eventprocessing type based needs of all the children and their decedents; (5)optionally perform any post-children processing of the event; (6) setthe binary flags of event type classifications that are now to behandled by this event processing unit in the future; (7) return controlto a parent processing unit if the parent reference is a first state(non-null state); and (8) return control to a main processing loop whichlogically called this processing unit if the parent reference is asecond state (null state).

In another embodiment (12) , the invention provides a method as in (11), wherein the optional pre-children and or post-children processing ofthe event is based on at least one of: (i) the event type; (ii) theevents fields, parameters and/or associated file contents values, and/orthose on any list of referenced child processing units to the tasks ofthe processing unit; and (iii) the current run-time environment asspecified by the event, the parent, and or the state of the device.

In another embodiment (13) , the invention provides a method as in (11), wherein the set up of any environmental changes needed may optionallyinclude at least one of providing a bitmap as a virtual screen,distorting the child's sense of time, distorting the coordinates of amouse click to conform to the different size, translation or angle oforientation the parent is displaying of the image or bitmap the childgizmo is rendering into.

In another embodiment (14) , the invention provides a method as in (11), wherein managing the links is used to flexibly and dynamically mix andmatch event processing units in a manner where changes in functionalityare easily made by creating and maintaining the ordered graph.

In another embodiment (15) , the invention provides a method as in (11), wherein where the environment for linked processing units can becontrolled by a parent processing unit to provide advanced functionalitywithout the need for the child processing units to understand how theirenvironments are being virtualized by parent processing units.

In another embodiment (16) , the invention provides a method as in (11), wherein the deterministic flow of processing makes for a robust systembecause the order of event processing is more tightly controlled andmore easily tested than they would be in non-deterministic asynchronousevent processing runtimes.

In another embodiment (17) , the invention provides a method as in (11), wherein processing units compiled into one Dart can proxy forseparately compiled and generated Darts that can dynamically be added tothe Linear Tasking graph and then function as if they were part of theoriginally generated Dart.

In another embodiment (18) , the invention provides a method as in (17), wherein separately generated Darts can be saved automatically as partsof parent Darts as the result of a SAVE_INSTRUCTION orSAVE_BUILTIN_INSTRUCTION.

In another embodiment (19) , the invention provides a method as in (1) ,wherein one or more or any combination of the following are true: (i)the runtime is the DartRuntime; (ii) the software object orientedframework is the DartFramework; (iii) the base event processing unitsare Dart Gizmos; (iv) the event processing unit classes which inheriteither directly or indirectly from the base event processing unit classincludes either the Dart Rendition class and/or the Dart MasterRenditionclass.

In another embodiment (20) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for ordering and managing event driven execution and runtimeenvironments of a plurality of event processing units of a softwareapplication package, the program module including instructions for:generating or providing a software object oriented framework whichincludes a base event processing unit class, and zero or more eventprocessing unit classes which inherit either directly or indirectly fromthe base event processing unit class; creating, maintaining, adding,deleting or reordering links that form a graph or topology of eventprocessing units in a manner that ensures that there is always a singlelinear deterministic ordering for passing events through the graph ofprocessing units formed by the links; and dynamically changing the graphor topology of processing units according to the needs of the runningapplication.

In another embodiment (21) , the invention provides a computer programproduct as in (20) , wherein the event processing unit base classcomprises at least one of the following class object members: a possiblynull reference to a parent processing unit; a possibly empty orderedlinear list of references to child processing units; one or moreoptional binary flag(s) corresponding to classifications of event typesthat are to be acted upon by the processing unit itself), wherein theflags are used to eliminate unnecessary passing of events down tochildren or their descendants that will not process any events of aparticular class; one or more child optional binary flag(s)corresponding to the classes of event types that are to be acted upon byany of the child processing units descending down a chain of parentprocessing and child processing until there are no more children orchild processing units; a procedure for adding, deleting, modifying,and/or reordering a reference to at least one of the parent and childprocessing units; and a procedure for ordering the processing of events.

In another embodiment (22) , the invention provides an apparatusproviding ordering and managing of event driven execution and runtimeenvironments of a plurality of event processing units, the apparatuscomprising: a processing logic or processor and a memory coupled withthe processing logic or processor; means, accessible to the processinglogic or processor, for generating or providing a software objectoriented framework which includes a base event processing unit class,and zero or more event processing unit classes which inherit eitherdirectly or indirectly from the base event processing unit class; means,accessible to the processing logic or processor, for creating,maintaining, adding, deleting or reordering links that form a graph ortopology of event processing units in a manner that ensures that thereis always a single linear deterministic ordering for passing eventsthrough the graph of processing units formed by the links; and means,accessible to the processing logic or processor, dynamically changingthe graph or topology of processing units according to the needs of therunning application.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

X. Vertical Layering

In another aspect of the invention, system, apparatus, method andcomputer program for vertical layering are provided. Vertical Layeringincludes enabling efficient and effective implementation of featureswhich by their nature involve close cooperation amongst the tool,application, framework, engine, Rendition and instruction-setimplementation, and operations.

In conventional software eco-systems or environments, the nativeinstruction set of the processor is abstracted in a number of horizontallayers which often include a Basic Input-Output System (BIOS), anoperating system, a graphics subsystem, an application framework, andfinally an application. In each case there is a nearly completeabstraction of what the layer is to do, implemented in the abstractionof the layer below. This mapping from abstraction layer to abstractionlayer conventionally requires a great deal of computer program code andobscures much of the information in one layer from those layers aboveand below them. A widely known and used example is the seven layers ofthe OSI model which are Physical, Data Link, Network, Transport,Session, Presentation, and finally Application.

While building devices and operating software in standardizedabstraction layers can be very useful in allowing code to be developedindependently from different entities, and still work together ongeneral purpose computers; there are problems which render the use ofmany abstractions levels especially undesirable when dealing with deviceinteroperability.

One problem of conventional horizontal layering of abstraction layers asin the OSI model, is that the complexity of the specifications leadsalmost inevitably to imperfect implementations which can interact withother perfect or imperfect implementation to produce a wide array oferrors when operating in cooperation with other layers orimplementations of the same layer. While the interactions of the layersresiding on a single computer are easily tested together andincompatibilities removed before distribution, the testing of layers andcorrections of incompatibilities found on devices which must work with alarge number of permutations of layers on other devices found in thefield is very difficult. Still more difficult is the task wherediffering horizontal layer implementations must cooperate across ad-hocteams of devices where the functions, resources, operating systems, CPUsor other processors or processing logic are all potentially different.

It would be rare for independently implemented layers to work the firsttime they were tested together on a single device, but through repeatedtesting and fixing of incompatibilities the layers of originallyindependently developed modules can be made to work reliably together ona single computer. The same repeated testing and fixing ofincompatibilities is not easily realizable for the potentially largenumber of permutations of devices and layer implementations which mayneed to interoperate. The result is that many incompatibilities are notfound until the devices using the separate implementations are inwidespread use.

Another problem with the traditional horizontal layering of abstractionlevels is that all interactions of all abstraction layers must be knownand understood by the time the standard is to be implemented in productsto ensure that all necessary interactions of layers can be passedthrough all intervening layers. An example, as will be explained furtherbelow, is the optional but very advantageous need for an application tointeract with the hardware to carry out efficient power management.Current horizontally layered protocol specification standards lack anysupport for the application to communicate its response timerequirements through all the intervening layers to the hardware. Whilethere are many power management implementations on devices, theseimplementations are most often highly device specific implementationsthat bypass or corrupt the standard implementation of the layers, anduse heuristics based on patterns of hardware access at the lower levelsin place of the actual response time needs knowable only to the softwarerunning at the higher application levels.

Alternatively to the conventional horizontal layering in the currentstate of the art, is the inventive Vertical Layering, which can beadvantageously employed if the programming tools, applications, players,runtime, operating system, and/or low-level functions, can share datastructures and communications semantics throughout.

In one implementation embodiment, Vertical Layering in optionally butadvantageously embodied throughout the implementation of theDartPlatform (FIG. 3). Two examples of where Vertical Layering isadvantageously employed in the DartPlatform, are power management andapplication level error recovery (FIG. 19).

For example, power management functions are most often implemented inconventional devices using heuristics based on input or outputactivities detected at the lowest levels; yet, only the applicationitself really knows if it requires control to return to its processingto decode the next frame of a video in 1/30 second, or there will be norequirement for further processing until the user does something.

In one advantageous embodiment of the invention, the dart platform(DartPlatform) collects the processor response requirements as the Dartruntime (DartRuntime) makes each pass though a single hierarchy LinearTasking of event processing units called Gizmos (FIG. 11, FIG. 15).After each pass through the intra-device parts of the DartRuntime (FIG.15), whenever a Dart is built using the Dart framework (DartFramework)hierarchy of Gizmo based processing units (FIG. 11), the DartPlatformcombines the minimum synchronous Gizmo tree response time requirements(FIG. 15 8010) with the asynchronous event time requirements in theevent queue (EventQueue FIG. 22 660) of the device engine (DartEngineFIG. 22 600) to determine an accurate and reliable amount of time thatthe device can power down or reduce power levels or consumption forbefore returning control to the DartPlayer. Methods of power management,such as reduction in a processor logic clock, reducing logic or powersupply voltage levels, or powering down portions of a larger circuit,are known in the art. Consider for example, an interoperability slideshow Dart application described herein before. This application runningon an originating device can form a team of say five devices throughRecruitment where each device in the team displays the same slide as theoriginating application, albeit optionally scaled or otherwise adaptedfor efficient transmission and display on each connection and eachdevice. What happens if a device with a wireless connection goes out ofrange, or a portable device shuts down automatically to save batterylife. In a conventional implementation only the slide show (SlideShow)application knows how to recover should this device suddenly move backin range or get turned back on (see example of FIG. 19). It wouldrequire a great deal of programming at all levels of a conventionalhorizontally layered device to be able to seamlessly recover, especiallywhere a non-originating device has powered off and lost the entire slideshow data and state. This is because conventional software eco-systemsdo not built into the horizontal abstractions the semantics necessaryfor all levels from the low-level routines that detect the lostconnections to the application. Without a direct mechanism forinformation to flow from the application to the low-level communicationssystem and back it is difficult for an application to be written toautomatically resend the data and state information to the connectionrecovered device.

With the Vertical Layering employed in the DartPlatform, resendingapplication data and procedures to recover an application over a team ofdevices when connections are lost is built into the application throughthe DartFramework, the DartRuntime, and the DartEngine. So anyapplication built using the DartFramework and run on a DartPlayer doesnot need any special programming for robust multi-device application anddata synchronization recovery.

In one embodiment of the Dart Platform, the traditionally low-leveloperations (such as for example, communications) interoperate directlywith the traditionally high-level operations (such as for example, theapplication) directly by passing DartEvents directly between the Dartapplication that is running on the DartInstructionSet and the nativecode that is handing communications.

The DartTools (FIG. 3 200, FIG. 12 200), DartInstructionSet, DartRuntime(FIG. 9, FIG. 15, FIG. 16, FIG. 17) and Linear Tasking (FIG. 18) are alldesigned to provide an environment largely devoid of layers ofabstraction where the applications form and process DartEvents with theexact same (or substantially the same) structure and semantics as thecommunications functions inside the DartEngine. Thus, applications caneasily and effectively collect, synchronize and communicate all itsneeds and status with all other software operations that are part of theDartPlatform through the passing and processing of Events understood bymost all components of an interoperating system at every level, whetherthese components are part of an application, the engine executing theapplication, or interoperating parts of an application distributedthroughout a team of devices.

In one embodiment, the vertical layers provides for a system, method,and computer program for closely coupling the operations of softwareapplications and the device control software to foster a high-level ofcooperative functionality between the application and the device controlsoftware with low-complexity, low-processing requirements, fewapplication program interfaces, and few protocol interfaces. It alsoprovides a software run-time model which is largely event driven for allsoftware operations whether application or device control related; aswell as a set of event semantics, types, structures and operations onevents common to the applications and the low-level device controlsoftware. An event queue which drives the sequencing of synchronousapplication event processing and asynchronous application, device,communications and interoperability operations, are also provided. Thevertical layering approach also provides an instuction set or systemcalls which are used to manage the queuing, dequeuing and processing ofevents. An optional robust device interoperability communicationsruntime model may also be provided, where communications between devicesis maintained, error corrected, and when necessary reestablished withcooperation, but a with a small amount of disruption to applicationsrunning effectively across a team of interoperating devices. A systemfor Serialization and Synchronization of Events passing through acooperative team of devices is beneficially applied to keep allcomponents of an interoperable system of asynchronous operations tightlycoupled, and therefore reliable, simple to implement, efficient andsimple to use.

In another embodiment, the invention provides a method, computer programsoftware, device and system for directly coupling the operations ofsoftware applications and the device control software operations on adevice or set of communicating devices to foster a highly efficient andflexible degree of cooperative functionality between softwareoperations, whether on one device or across one or more cooperatingdevices. This may include source code and resources, software tools, aprepackaged or pre-specified software framework or library, an eventdriven runtime, and an instruction set or set of system calls to managea queue of events on each device. In at least one embodiment exactly onequeue of events is managed on each device for this purpose.

The application may be a Dart, and the device control software may be aDartEngine or DartPlayer. In some embodiments the set of devices is orincludes a team set up using Recruitment as described herein or via adifferent method for recruiting a team of devices. In one embodiment,the software framework is the DartFramework. In one embodiment the eventdriven runtime is the DartRuntime. In one embodiment the source code andresources are the DartSource source code and the software tools are theDartTools. In one embodiment, the instruction set and system calls areprovided by the DartInstructionSet and/or the functions that can beinvoked as part of built-in instruction type instructions, such as forexample the Dart BUILTIN_INSTRUCTION (FIG. 20 670, 671, 672, 673) and/orthe OEM_BUILTIN_INSTRUCTION. (FIG. 20, 674, 680, 681, 682).

In at least one embodiment, the efficiency is achieved at least in partby having a single data structure with commonly understood semanticsused to communicate information and/or content and/or status amongst thesystem of devices (FIG. 17 800), applications, and communications code.

In at least one embodiment, events such as for example, DartEvents (FIG.17 800), are used as the single data structure.

In at least one embodiment, the device control software operationsincludes communications, device configuration management, devicediscovery, managing allocation, de-allocation and access to memory,physical storage, physical displays physical input/output devices or anyother physical or software virtualized physical aspect of a device.

In one embodiment, software tools take the software source code andresources written to use the pre-specified software framework which willensure conformance to the event driven execution model of the runtimesupported by access to the instruction set or set of system calls whichare used to manage the enqueuing, processing and dequeuing of events.

In at least one embodiment, the event processing of the application isas expressed and described for FIG. 15, FIG. 9, FIG. 16, FIG. 17. In atleast one embodiment, the events put onto the queues of devices areserialized and synchronized. The serialization and synchronization areperformed according to the serialization and synchronization methodshown in FIG. 9.

Particular exemplary embodiments involving an aspect of verticallayering are now described. In one embodiment (1) , the inventionprovides an event driven vertical layering system for coordinating theoperations of and data movement between procedural (software) componentswhich perform application level operations, device hardware controllevel operations, communications level operations, or any other level orsubset of operations within or between one or more teamed devices toestablish an efficient and/or robust cooperative functionality betweenthe procedural (software) components, the system comprising: (a) astatic event data structure whose fields and field semantics aregenerally known and understood between all the event generating andprocessing units across one or more devices and the procedural(software) components running in the one or more devices; (b) a queue oneach teamed device which stores, removes, manages, and controls accessto the event data structure instances; (c) means for managing theplacing, modification, and removing of events on the queue accessiblefrom all the cooperating procedural (software) components; (d) means forspecifying and maintaining a common list of event types which are to beserialized and synchronized between the queues of the cooperatingdevices; and (e) means for ensuring that all the events of any of thetypes on the common list are processed by the procedural (software)components in the exact same order on all devices regardless of whatprocedural (software) components initiated the events, or which of theteamed devices the procedural (software) components that initiated theevents are running on.

In another embodiment (2) , the invention provides the system of (1) ,wherein the queue on each teamed device which stores, removes, manages,and controls access to the event data structure instances consists ofexactly one queue on each teamed device.

In another embodiment (3) , the invention provides the system of (1) ,wherein the means for managing the placing, modification, and removingof events on the queue accessible from all the cooperating procedural(software) components comprises a procedure implemented as a computerprogram including a plurality of program instructions executing in aprocessor logic of the interoperating devices.

In another embodiment (4) , the invention provides the system of (1) ,wherein the means for specifying and maintaining a common list of eventtypes which are to be serialized and synchronized between the queues ofthe cooperating devices comprises a procedure implemented as a computerprogram including a plurality of program instructions executing in aprocessor logic of the interoperating devices.

In another embodiment (5) , the invention provides the system of (1) ,wherein the means for ensuring that all the events of any of the typeson the common list are processed by the procedural (software) componentsin the exact same order on all devices regardless of what procedural(software) components initiated the events, or which of the teameddevices the procedural (software) components that initiated the eventsare running on comprises a procedure implemented as a computer programincluding a plurality of program instructions executing in a processorlogic of the interoperating devices.

In another embodiment (6) , the invention provides the system of (1) ,wherein one or more or any combination of the following are true: (1)the static event data structure is the DartEvent; (2) the teamed deviceswhere assembled for cooperation through the used of device recruitment;(3) the methods for managing the placing, modification and removing ofevents are accessed through the use of an Interoperability InstructionSet or the DartInstructionSet; (4) the system carries out its eventdriven functionality at least in part through the InteroperabilityRuntime or the DartRuntime; (5) the methods for specifying the commonlist and ensuring that all events of any of the types on the common listare processed in the same order are as described for serialization andsynchronization of events on Recruitment; and (6) the InteroperabilityTools or the DartTools are used to generate the software applicationoperations code of the system.

In another embodiment (7) , the invention provides the system of (1) ,wherein coordinating the operations is a means for directly coupling theoperations of the procedural (software) components without the need forany intermediating layers of software, application program interface(API), or other semantics translating means or mechanism.

In another embodiment (8) , the invention provides the system of (7) ,wherein the intermediating layers of software, API, or other semanticstranslating means or mechanism which are not needed as intermediatinglayers includes any of the seven horizontal layers of the conventionalOSI Protocol model, or combinations thereof.

In another embodiment (9) , the invention provides a system as in (1) ,wherein at least a component of the efficiency is achieved by having asingle data structure with commonly understood semantics used tocommunicate and transport information and/or content and/or statusamongst the devices and procedural (software) components.

In another embodiment (10) , the invention provides a system as in (7) ,wherein at least a component of the efficiency is achieved by having nointermediating layers of software which needs to process thecommunication or transport.

In another embodiment (1) , the invention provides a system as in (7) ,wherein at least a component of the robustness is achieved by having nointermediating layers of software where problems or incompatibilities ormisunderstandings of implementation may otherwise occur.

In another embodiment (12) , the invention provides a system as in (1) ,wherein application level operations optionally include one or more orany combination of the following: (1) application directed powermanagement; (2) application directed error recovery; (3) applicationdirected Interfacing and or any interactively with a human or automateduser; (4) application directed media rendering or editing; and (5)application directed data processing.

In another embodiment (13) , the invention provides a system as in (12), wherein application directed means that the initiation and or carryingout of operations is performed at least in part by the processinginstructions generated by traditional compilers and linkers.

In another embodiment (14) , the invention provides a system as in (12), wherein application directed means that the initiation and or carryingout of operations is performed at least in part by the processinginstructions of an interoperability instruction set generated by one ormore Interoperability Tools or by one or more DartTools.

In another embodiment (15) , the invention provides a system as in (12), wherein the application conforms to the Interoperability Format or theDartFormat.

In another embodiment (16) , the invention provides a system as in (1) ,wherein device hardware control level operations optionally includes oris selected as one or more of the following operations: (1) accessingone or more in any combination of memory, hard disk drive storage,displays, power regulation, device power-up, and device shutdown; (2)compression/decompression circuits or processors, or input and outputcircuits of any type; (3) setting or reading hardware operating modes orany other information physically stored digitally or in an analogfashion; (4) accessing random numbers or other forms of entropy for usein simulations or cryptographic and or security operations; (5) managingdevice configuration; (6) discovering devices and/or services; and (7)managing allocation and/or deallocation and or access to memory,physical storage, physical displays physical input/output devices, orany other physical or software virtualized physical aspect of a device,and any combinations of these.

In another embodiment (17) , the invention provides a system as in (1) ,wherein communications level operations optionally includes or isselected as any one or more of the following in any combination: (1)sending and or receiving of data, and or code and or content; (2)sending and or receiving of meta data about data, and/or code and/orcontent to be sent or received; (3) device and/or service discovery; (4)broadcasting of data and/or code and/or content; (5) transmission errordetection and or correction; (6) establishing and or maintainingchannels between devices; (7) establishing and or maintaining separatelogical sessions on a single and or multiple channels; (8) managing anyof the following: (a) bridging protocols; (b) extending the reach ofcommunications mechanisms; (c) acting as a proxy for a device; and (d)providing a gateway for passing information through or from differentphysical media or protocols.

In another embodiment (18) , the invention provides a system as in (1) ,wherein an Interoperability Engine or a DartEngine or a DartPlayer orany combination of these is used at least in part to embody the system.

In another embodiment (19) , the invention provides a system as in (1) ,further comprising software tools adapted for operating on softwaresource code and resources written to use a particular software frameworkwhich will ensure conformance to the event driven execution model of theruntime supported by access to an instruction set or to a set of systemcalls which are used to manage the enqueuing, processing, and dequeuingof events.

In another embodiment (20) , the invention provides a system as in (19), wherein one or more of the following in any combination are true: (1)the software tools comprise the Interoperability Tools or the DartTools;(2) the software framework comprises the Interoperability Framework orthe DartFramework; (3) the event driven execution model comprises theInteroperability Runtime or the DartRuntime; and (4) the instruction setor set of system calls comprises the Interoperability Instruction Set orthe DartInstructionSet.

In another embodiment (21) , the invention provides an event drivenvertical layering method for coordinating the operations of and datamovement between procedural components within or between one or moreteamed devices, the method comprising: (a) defining or generating astatic event data structure whose fields and field semantics aregenerally known and understood between all the event generating andprocessing units across one or more devices and the procedural(software) components running in the one or more devices; (b) definingor generating a queue on each teamed device which stores, removes,manages, and controls access to the event data structure instances; (c)managing the placing, modification, and removing of events on the queueaccessible from all the cooperating procedural (software) components;(d) specifying and maintaining a common list of event types which are tobe serialized and synchronized between the queues of the cooperatingdevices; and (e) ensuring that all the events of any of the types on thecommon list are processed by the procedural components in the exact sameorder on all devices regardless of what procedural components initiatedthe events, or which of the teamed devices the procedural componentsthat initiated the events are running on.

In another embodiment (22) , the invention provides an event drivenvertical layering method as in (21) , wherein the procedural componentsperform application level operations, device hardware control leveloperations, communications level operations, or any other level orsubset of operations within or between one or more teamed devices toestablish an efficient and/or robust cooperative functionality betweenthe procedural components.

In another embodiment (23) , the invention provides an event drivenvertical layering method as in (21) , wherein the procedural componentsare implemented as computer program code software.

In another embodiment (24) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for coordinating the operations of and data movement betweenprocedural components within or between one or more teamed devices, theprogram module including instructions for: (a) defining or generating astatic event data structure whose fields and field semantics aregenerally known and understood between all the event generating andprocessing units across one or more devices and the procedural(software) components running in the one or more devices; (b) definingor generating a queue on each teamed device which stores, removes,manages, and controls access to the event data structure instances; (c)managing the placing, modification, and removing of events on the queueaccessible from all the cooperating procedural (software) components;(d) specifying and maintaining a common list of event types which are tobe serialized and synchronized between the queues of the cooperatingdevices; and (e) ensuring that all the events of any of the types on thecommon list are processed by the procedural components in the exact sameorder on all devices regardless of what procedural components initiatedthe events, or which of the teamed devices the procedural componentsthat initiated the events are running on.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XI. Application Event Driven Power Management

Recall that Dart applications built using LinearTasking and orVerticalLayering as embodied at least partially in the DartFramework,always keep track of their exact response time needs so that efficientpower management techniques such as slowing down the processor canextend the lifetime of batteries, limit the amount of energy consumed orlimit the amount of heat generated on devices. In the current state ofthe art most applications do not keep track of their response timeneeds, and if they did would not be able to communicate these needsthrough existing layers of protocols which conform to specificationsthat do not include interfaces for communicating response time needs tothe hardware of the device from the application.

Particular embodiments of application event driven power management arenow described. Additional embodiments are set forth in the examples. Inone embodiment (1) , the invention provides a method for device energyand power management and energy and power consumption reductioncomprising: establishing an event driven runtime environment within atleast one device for which the energy and power management andconsumption reduction is to be achieved; generating and maintaining atleast one event queue identifying all synchronous and asynchronousprocessing events and associated minimum expected processing times forthe processing events in the queue; and selecting a final minimumestimated response time based on the expected minimum synchronous andasynchronous event processing times in the queue.

In another embodiment (2) , the invention provides a method as in (1) ,further comprising: searching through all the events on a firstasynchronous event queue that drive all the asynchronous processing andcollect the minimum time needed before any of the asynchronous eventsneed to be processed; searching through all the events on a secondsynchronous event queue that drive all the synchronous processing andcollect the minimum time needed before any of the synchronous eventsneed to be processed; and determining the final minimum response timeneeded by selecting the lesser of the minimum time needed for theasynchronous events and the minimum time needed for the synchronousevents.

In another embodiment (3) , the invention provides a method as in (2) ,further comprising: performing power management or power consumptionreduction tasks using the final minimum response time value beforefurther processing any events.

In another embodiment (4) , the invention provides a method as in (2) ,wherein the first asynchronous event queue and the second asynchronousevent queue are the same single unified event queue.

In another embodiment (5) , the invention provides a method as in (2) ,wherein the first asynchronous event queue and the second asynchronousevent queue are different event queues.

In another embodiment (6) , the invention provides a method as in (3) ,wherein the events are DartEvents.

In another embodiment (7) , the invention provides a method as in (3) ,wherein both synchronous and asynchronous events are maintained on asingle unified queue.

In another embodiment (8) , the invention provides a method as in (7) ,wherein the unified queue is managed by an Interoperability Engine asdescribed elsewhere in this specification or by the DartEngine.

In another embodiment (9) , the invention provides a method as in (3) ,wherein the event driven runtime is an Interoperability Runtime or is aDartRuntime.

In another embodiment (10) , the invention provides a method as in (3) ,wherein the synchronous events are queueable events which drive thesynchronous processing needed to carry out the intent of the applicationby getting processed by the event processing units of an applicationthat is compiled and linked.

In another embodiment (11) , the invention provides a method as in (3) ,wherein asynchronous events are queuable events which drive theasynchronous processing needed to carry out the intent of anapplication, drive state machine driven hardware functions, or create,maintain, or make use of the communications of one device with anotherby getting processed by an Interoperability Engine or a DartEngine.

In another embodiment (12) , the invention provides a method as in (3) ,wherein the power management and/or power consumption reduction tasksare carried out by one or more of the following in any combination: (a)making a native threaded operating system call to initiating theblocking of the current thread of execution until such time as anexternal stimulus or an explicit timeout based on the final minimumresponse time needs, or an implied or device specific timeout period hasexpired. (b) returning control to a calling cooperative processes whichis given the time period for which no processing by the events on thequeue or queues in needed. (c) directly or indirectly instructing thehardware and or processor to slow down, speed up, or shut down itsclock(s) or various hardware units. (d) directly or indirectlyinstructing device hardware and/or processor and/or logic to control theamount of voltage and or current in one or more circuits or electricalunits; and (e) any other method wherein power use is regulated based onthe final minimum response time.

In another embodiment (13) , the invention provides a method for devicepower management and/or power consumption reduction carried out at leastin part by an event driven runtime, the method comprising: (1) at thestart of each processing pass through an event driven application,setting a minimum response time variable to its expected highest-value,which expected highest-value is interpreted by the system as a valueindicating an infinite response time need of the application eventprocessing and any asynchronous event processing driven by eventsalready in the runtime event queue; (2) when the first synchronous eventprocessing unit of an application is called, the first synchronous eventprocessing unit checking to determine if the event reference passed intothe processing unit as a parameter reference points to an actual eventinstance to be processed; (3) if there is no event referenced, theprocessing unit makes a system call directly or indirectly by way ofexecuting an instruction to request the next synchronous applicationevent that needs processing, the system call causing the run-time tolook through all the asynchronous operations that are being driven byevents on the queue, and if there is an event to be processed, thenprocessing continues at step 8. (4) asynchronous events on the queuewhich are ready for processing cause an operation corresponding to eachasynchronous event to be executed; (5) as each asynchronous event on thequeue is inspected to see if it is ready to be processed, the minimumtime until the next asynchronous event will be ready to be processed isgathered in the minimum response time variable, replacing the highestwait time value with any gathered minimum time value which is less thanthe highest value; (6) when the system call returns, a return value istested to determine whether there is a synchronous application event nowat the head of the queue which needs to be processed by the application;(7) if there is no synchronous application event at the head of thequeue which needs to be processed, then a synchronous applicationprocessing event of a type to indicate general processing is generatedfor application processing; (8) the synchronous application event isinspected by all application event processing units in a predefinedorder according to the software run-time model; (9) any synchronousapplication event processing unit that needs to continue processingafter it processes the current synchronous event will change the minimumresponse time variable value to the minimum of the minimum response timevariable value and that of its own continued processing unit needs; (10)when the first synchronous event processing procedure regains control, adevice specific power management function is called with the minimumresponse time variable value as a parameter; and (11) the devicespecific power management function uses the value to stop, slow down,modify an operating voltage or other parameter of the power consumingcomponents of the device until an externally generated stimulus occursor the time specified it the parameter occurs.

In another embodiment (14) , the invention further including: searchingthrough all the events on a first asynchronous event queue that driveall the asynchronous processing and collect the minimum time neededbefore any of the asynchronous events need to be processed; searchingthrough all the events on a second synchronous event queue that driveall the synchronous processing and collect the minimum time neededbefore any of the synchronous events need to be processed; anddetermining the final minimum response time needed by selecting thelesser of the minimum time needed for the asynchronous events and theminimum time needed for the synchronous events.

In another embodiment (15) , the invention provides the method of (2) ,further comprising: (1) at the start of each processing pass through anevent driven application, setting a minimum response time variable toits expected highest-value, which expected highest-value is interpretedby the system as a value indicating an infinite response time need ofthe application event processing and any asynchronous event processingdriven by events already in the runtime event queue; (2) when the firstsynchronous event processing unit of an application is called, the firstsynchronous event processing unit checking to determine if the eventreference passed into the processing unit as a parameter referencepoints to an actual event instance to be processed; (3) if there is noevent referenced, the processing unit makes a system call directly orindirectly by way of executing an instruction to request the nextsynchronous application event that needs processing, the system callcausing the run-time to look through all the asynchronous operationsthat are being driven by events on the queue, and if there is an eventto be processed, then processing continues at step 8 of the procedure;(4) asynchronous events on the queue which are ready for processingcause an operation corresponding to each asynchronous event to beexecuted; (5) as each asynchronous event on the queue is inspected tosee if it is ready to be processed, the minimum time until the nextasynchronous event will be ready to be processed is gathered in theminimum response time variable, replacing the highest wait time valuewith any gathered minimum time value which is less than the highestvalue; (6) when the system call returns, a return value is tested todetermine whether there is a synchronous application event now at thehead of the queue which needs to be processed by the application; (7) ifthere is no synchronous application event at the head of the queue whichneeds to be processed, then a synchronous application processing eventof a type to indicate general processing is generated for applicationprocessing; (8) the synchronous application event is inspected by allapplication event processing units in a predefined order according tothe software run-time model; (9) any synchronous application eventprocessing unit that needs to continue processing after it processes thecurrent synchronous event will change the minimum response time variablevalue to the minimum of the minimum response time variable value andthat of its own continued processing unit needs; (10) when the firstsynchronous event processing procedure regains control, a devicespecific power management function is called with the minimum responsetime variable value as a parameter; and (11) the device specific powermanagement function uses the value to stop, slow down, modify anoperating voltage or other parameter of the power consuming componentsof the device until an externally generated stimulus occurs or the timespecified it the parameter occurs.

In another embodiment (16) , the invention provides the method of (13) ,wherein the synchronous events are processed by Dart Gizmo classinstances or instances of classes that inherit directly or indirectlyfrom the Dart Gizmo class.

In another embodiment (17) , the invention provides the method of (15) ,wherein the synchronous events are processed by Dart Gizmo classinstances or instances of classes that inherit directly or indirectlyfrom the Dart Gizmo class.

In another embodiment (18) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for device energy and power management and energy and powerconsumption reduction of the computer system or information appliance,the program module including instructions for: establishing an eventdriven runtime environment within at least one device for which theenergy and power management and consumption reduction is to be achieved;generating and maintaining at least one event queue identifying allsynchronous and asynchronous processing events and associated minimumexpected processing times for the processing events in the queue; andselecting a final minimum estimated response time based on the expectedminimum synchronous and asynchronous event processing times in thequeue.

In another embodiment (19) , the invention provides a computer programproduct as in (18) , further comprising instructions for: searchingthrough all the events on a first asynchronous event queue that driveall the asynchronous processing and collect the minimum time neededbefore any of the asynchronous events need to be processed; searchingthrough all the events on a second synchronous event queue that driveall the synchronous processing and collect the minimum time neededbefore any of the synchronous events need to be processed; anddetermining the final minimum response time needed by selecting thelesser of the minimum time needed for the asynchronous events and theminimum time needed for the synchronous events.

In another embodiment (20) , the invention provides a computer programproduct as in (19) , further comprising instructions for: performingpower management or power consumption reduction tasks using the finalminimum response time value before further processing any events.

In another embodiment (21) , the invention provides an apparatus forenergy and power management and energy and power consumption reductionwithin at least one device, the apparatus comprising: a logic circuitwithin the at least one device utilizing energy and power to perform alogic processing operation; means for establishing an event drivenruntime environment within at the least one device for which the energyand power management and consumption reduction is to be achieved; meansfor generating and maintaining at least one event queue identifying allsynchronous and asynchronous processing events and associated minimumexpected processing times for the processing events in the queue; meansfor selecting a final minimum estimated response time based on theexpected minimum synchronous and asynchronous event processing times inthe queue; and means for searching through all the events on a firstasynchronous event queue that drive all the asynchronous processing andcollect the minimum time needed before any of the asynchronous eventsneed to be processed; means for searching through all the events on asecond synchronous event queue that drive all the synchronous processingand collect the minimum time needed before any of the synchronous eventsneed to be processed; means for determining the final minimum responsetime needed by selecting the lesser of the minimum time needed for theasynchronous events and the minimum time needed for the synchronousevents; and means for performing power management or power consumptionreduction tasks for the logic circuit using the final minimum responsetime value before further processing any events.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XII. Interoperability Application Event Driven Error Recovery

Recall that device to device wireless communications connections areoften unreliable due to interference, distance limitations, and abruptshutdowns due to low battery power. In conventional horizontally layeredprotocol software implementations on devices a fatal error in any onelayer will result in unrecoverable errors which will be difficult for anapplication to recover from, both because the application does not havestandard interfaces to easily reestablish the connections and contextsof the connections, and because conventional application programs do nothave much infrastructure for tracking and reestablishing shared statebetween applications running on different devices. The DartFrameworkkeeps track of shared state, renditioning can be used to easilyreestablish lost state between devices and VerticalLayering makes itsimple for communications errors to be relayed to the Dart and for theDart to relay recovery information directly to the communicationsprocessing units. Thus Darts running across devices can seamlesslyrecover from intermittent complete losses of communications betweencooperating devices and the recovery of the shared state of the deviceswhen the connection is restored even where the previously lost devicehas itself lost all its application state.

Additional embodiments of Interoperability application event drivenerror recovery are now described. In one embodiment (1) , the inventionprovides a method for gracefully continuing an operation in anenvironment of lost or intermittent communications), wherein an eventdriven interoperability application package running cooperatively acrossmultiple teamed devices gracefully continues its operations and/orpartially recovers from temporarily loosing communications with devicesthat are part of the team, the method comprising: (1) generating acommunications session lost type event instance on a team member devicewhen communication from the team member device to a teamed device islost or interrupted and cannot be reestablished within a predeterminedor dynamically determined time period and wherein each team member andteamed device is carrying out part of the intent of an applicationpackage; (2) sending the communications session lost type event isdirectly or via a queue of events which drives synchronous operations ofthe application package to the application event processing unit whichhandles communications session lost events;

-   -   (3) the event processing unit of the application package        modifying the behavior of the application package where possible        to continue its operations without the teamed device with which        communications has been lost or interrupted; (4) generating a        communications session recovered type event instance on the team        member device when the communication is restored between the        team member and teamed devices; (5) the communications session        recovered type event being sent directly or via a queue of        events which drives the synchronous operations of the        application to the application event processing unit which        handles recovered communications sessions; (6) the event        processing unit of the application then causing the team member        device to send whatever code, data, and/or content is needed to        bring the teamed device into synchronization with the rest of        the event driven interoperability application; and (7) the event        processing unit of the application then modifying the behavior        of the application package to include the now recovered teamed        device in the process of carrying out the intent of the        application.

In another embodiment (2) , the invention provides the method of (1) ,wherein one or more of the following are true in any combination: (1)the event driven interoperability application package conforms to theinteroperability format or is the DartFormat; (2) the interoperabilityapplication is as described elsewhere in this detailed description orthe interoperability package is a Dart; (3) the recruitment method isused to establish the running cooperatively across multiple teameddevices; (4) the events are or include DARTEvents; (5) the eventprocessing unit is a Dart Gizmo class instance, or an instance of anyclass which inherits directly or indirectly from the Dart Gizmo class;(6) the coordination and synchronization of events is carried out oneach device by an interoperability engine or is carried out by theDartEngine; and (7) the coordination and synchronization of events onand between devices is carried out according to an interoperabilityruntime or the DartRuntime.

In another embodiment (3) , the invention provides the method of (1) ,wherein the modified behavior of the application to continue itsoperations without the teamed device is optionally selected from the setof modifications consisting of one or more of the following in anycombination: (1) the device which is lost is simply displaying statusallowing the application to continue without the display; (2) thefunctions being done by the lost device are redundant and are thereforenot required to carry out the intent of the application; (3) thefunctions being done by the lost device can be performed instead by anyremaining member or members of the team; (4) the functions being done bythe lost device can be performed instead by another device reachable bythe team, and which can then be recruited into the team; (5) thefunctionality of the remaining teamed devices can proceed in a reducedmode of operation; and (6) any combination of the above.

In another embodiment (4) , the invention provides the method of (1) ,wherein the connection is lost due to any one or more of these factorsalone or in any combination: (1) battery power gets low or runs out andthe device shuts down its communications, or the entire device is shutdown, and or the device stops functioning; (2) a user or other automatedor non-automated process causes the device to stop functioning as partof the team without smoothly shutting down the application; (3) thedevice or communications protocol stops functioning correctly for anyreason other than an orderly shutdown of the part of the applicationrunning on the device; (4) the device is reset; (5) the device goes outof range of the wireless protocol being used; (6) the device looses lineof sight needed to maintain communication over a directional protocol;(7) interference by signals or objects which block the communications;and (8) any combination of the above.

In another embodiment (5) , the invention provides the method of (1) ,wherein a hierarchy of event processing units comprise a hierarchy ofprocessing units that include at least one Dart Gizmo processing unit ormany Dart Gizmo processing units arranged in a graph with processingsequenced and coordinated using linear tasking.

In another embodiment (6) , the invention provides the method of (1) ,wherein continuing or recovering is performed over a team of cooperatingdevices when connections are lost is carried out through the use of theDartFramework, the DartRuntime, and/or the DartEngine.

In another embodiment (7) , the invention provides the method of (6) ,wherein Darts built using the DartFramework and run on a Dart Player donot need any application specific special programming to support robustmulti-device application, data, and communications session recovery.

In another embodiment (8) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for gracefully continuing an operation in a processingenvironment of lost or intermittent communications), wherein an eventdriven interoperability application package running cooperatively acrossmultiple teamed devices gracefully continues its operations and/orpartially recovers from temporarily loosing communications with devicesthat are part of the team, the program module including instructionsfor: (1) generating a communications session lost type event instance ona team member device when communication from the team member device to ateamed device is lost or interrupted and cannot be reestablished withina predetermined or dynamically determined time period and wherein eachteam member and teamed device is carrying out part of the intent of anapplication package; (2) sending the communications session lost typeevent is directly or via a queue of events which drives synchronousoperations of the application package to the application eventprocessing unit which handles communications session lost events; (3)the event processing unit of the application package modifying thebehavior of the application package where possible to continue itsoperations without the teamed device with which communications has beenlost or interrupted; (4) generating a communications session recoveredtype event instance on the team member device when the communication isrestored between the team member and teamed devices; (5) thecommunications session recovered type event being sent directly or via aqueue of events which drives the synchronous operations of theapplication to the application event processing unit which handlesrecovered communications sessions; (6) the event processing unit of theapplication then causing the team member device to send whatever code,data, and/or content is needed to bring the teamed device intosynchronization with the rest of the event driven interoperabilityapplication; and (7) the event processing unit of the application thenmodifying the behavior of the application package to include the nowrecovered teamed device in the process of carrying out the intent of theapplication.

In another embodiment (9) , the invention provides an apparatus capableof gracefully continuing an operation in an environment of lost orintermittent communications with another teamed device), wherein anevent driven interoperability application package running cooperativelyacross multiple teamed devices gracefully continues its operationsand/or partially recovers from temporarily loosing communications withother teamed devices that are part of the team, the apparatuscomprising: a processor or processing logic and a memory coupled withthe processor or processing logic; logic means generating acommunications session lost type event instance on a team member devicewhen communication from the team member device to a teamed device islost or interrupted and cannot be reestablished within a predeterminedor dynamically determined time period and wherein each team member andteamed device is carrying out part of the intent of an applicationpackage; first communications means sending the communications sessionlost type event is directly or via a queue of events which drivessynchronous operations of the application package to the applicationevent processing unit which handles communications session lost events;an event processing unit of the application package for modifying thebehavior of the application package where possible to continue itsoperations without the teamed device with which communications has beenlost or interrupted; second communications means generating acommunications session recovered type event instance on the team memberdevice when the communication is restored between the team member andteamed devices; the communications session recovered type event beingsent directly or via a queue of events which drives the synchronousoperations of the application to the application event processing unitwhich handles recovered communications sessions; the event processingunit of the application causing the team member device to send whatevercode, data, and/or content is needed to bring the teamed device intosynchronization with the rest of the event driven interoperabilityapplication; and the event processing unit of the application thenmodifying the behavior of the application package to include the nowrecovered teamed device in the process of carrying out the intent of theapplication.

In another embodiment (10) , the invention provides the apparatus as in(9) , wherein the apparatus comprises at least one of a computer, amusic player, a media player, a personal data appliance (PDA), aninformation appliance, a printer, a recorder, a mobile or cellulartelephone, a camera, an electrical appliance, or any combination ofthese.

In another embodiment (11) , the invention provides the apparatus of (9), wherein one or more of the following are true in any combination: (1)the event driven interoperability application package conforms to theinteroperability format or is the DartFormat; (2) the interoperabilityapplication is a Dart; (3) the recruitment method is used to establishthe running cooperatively across multiple teamed devices; (4) the eventsare or include DARTEvents; (5) the event processing unit is a Dart Gizmoclass instance, or an instance of any class which inherits directly orindirectly from the Dart Gizmo class; (6) the coordination andsynchronization of events is carried out on each device by aninteroperability engine or is carried out by the DartEngine; and (7) thecoordination and synchronization of events on and between devices iscarried out according to an interoperability runtime or the DartRuntime.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XIII. Interoperability Instruction Set

In another aspect, the invention provides an interoperability method,software, and instruction set. Software or hardware that implement anInteroperability Instruction Set (IIS) is an efficient methodology forimplementing a common procedural environment as benefited to the: (i)Recruitment model for teaming and spreading or distributing anapplication; (ii) for allowing unmodified applications to run onotherwise dissimilar devices; and (iii) for exposing the uniqueresources, unique capabilities and/or unique content to otherapplications and devices.

In one advantageous embodiment and implementation of the invention, theinteroperability instruction-set is the Dart instruction set(DartInstructionSet) as embodied in or compatible with the Dart engine(DartEngine FIG. 22 600).

It will be appreciated that components of computer and informationsystems and devices may be implemented in hardware, firmware, and/orsoftware, and that there is often a design and implementation choice asto which of hardware, firmware, or software to use for a particularcomponent of a particular implementation. So it is with the Dart engineand the interoperability instruction set. Therefore it may beappreciated that although certain components of certain embodiments maybe described in terms of one of hardware, firmware, or software;alternative embodiments may use different combinations of hardware,firmware, or software to provide the means for accomplishing the desiredfunction or result.

In one exemplary embodiment, the hardware and/or software for carryingout an Interoperability Instruction-set is comprised of elevencomponents, though the components and functions may be groupeddifferently so that the number is not determinative of the structure oroperation of the engine or instruction set (see FIG. 4 3010).

First, a processor or central processing unit (CPU), memory, and inputoutput (I/O) capabilities for programs to run or execute on, and tosupport communications to and from systems, devices, networks and thelike outside or external to the device.

Second, there should be memory access, computation, test and branch,input/output instructions to carry out at least conventional generalpurpose computing tasks.

Third, there are advantageously provided interoperability performanceenhancing instructions used to extend the practical reach of commonbinary applications and Renditions to lower performance devices.

Fourth, there are advantageously provided interoperability instructionsto carry out the Dart methodologies of Recruitment, Renditioning,Creationism, Vertical Layering, Linear Tasking, Social Synchronization,Social Security, and VirtualPointers although as described elsewhereherein not all of these Dart components are required for all embodimentsof implementations.

Fifth, there are also advantageously provided certain unique capabilityinstructions that are operative to expose any characteristics,resources, capabilities, and/or functions possessed by or accessiblefrom of a particular device to software applications and other devices.

Sixth, security maintenance instructions are advantageously provided tocontrol and otherwise access the setting of security features, thegrouping of devices with a particular set of cross device access rights,and/or setting the access rights for applications to devices and/orresources.

Seventh, containment instructions, such as Dart containment(DartContainment) instructions, are also advantageously provided toallow Darts or Dart compatible instructions to effectively extend theexecution of an originating Dart across other separately generated Dartsthat are collected, maintained, and run as part of the operation of theoriginating Dart.

Eighth, common user-interface (UI) instructions are advantageouslyprovided for one or more of decoding, encoding, compressing anddecompressing and manipulating and rendering pictures, bitmaps, sounds,input events, text rendering, and other such operations.

Ninth, communications instructions are advantageously provided forexample, for opening, closing, and maintaining sessions and the datathat goes across the sessions.

Tenth, storage instructions are advantageously provided to control andmaintain access to storage, storage devices, storage resources, and thelike.

Eleventh, compatibility instructions are advantageously provided toconvert or transcode between differing formats or parameters of data,content and/or code.

The advantages of an Interoperability Instruction-set over other commonmethodologies, such as for example the use of virtual machines, is thatthe interoperability instruction set (and particularly the DartInteroperability Instruction Set) is designed and optimized to performall necessary interoperability operations as instructions that aredispatched to functions which are compiled or assembled into the nativecode of the device processor. In one embodiment that includes someoptional features and capabilities, the Interoperability Instruction-setshould have most if not all of the following: (ii) Recruitmentinstructions, (ii) Profile instructions, (iii) Synchronizinginstructions, (iv) user-interface or UI and graphics instructions, (v)power management instructions, (vi) Connection and Session managementinstructions, (vii) Storage instructions, (viii) Rendition instructions,(ix) Creationism instructions, (x) application parts managementinstructions, (xi) cryptographic large number math instructions, (xii)Text and/or symbol parsing instructions, (xiii) Virtual Pointermanagement instructions, and (xiv) instructions that are capable ofexposing unique capabilities of the device to DARTs and thereby to anyother DartDevices or Dart compatible devices.

Additional particular embodiments of the interoperability instructionset are now described. In one embodiment (1) , the invention provides anapparatus for effecting an interoperability instruction set (IIS), theapparatus comprising: a processor, a memory coupled to the processor,and an input/output (I/O) interface to support communications with theprocessor by an entity external to the apparatus; execution supportiveinteroperability means for carrying-out methodologies involving at leastone of recruitment, renditioning, creationism, vertical layering,linear-tasking, social synchronization, and social security; andcommunications interoperability instructions for opening and maintaininga communication session and the procedures, data, content or otherinformation that goes between communicating devices during thecommunication session.

In another embodiment (2) , the invention provides an apparatus as in(2) , wherein: the execution supportive interoperability means comprisesinteroperability instructions for carrying-out the methodologies.

In another embodiment (3) , the invention provides an apparatus as in(1) , wherein one or more of the following are true: recruitment is asdescribed elsewhere in this detailed description; renditioning is asdescribed elsewhere in this detailed description; creationism is asdescribed elsewhere in this detailed description; vertical layering isas described elsewhere in this detailed description; liner tasking is asdescribed elsewhere in this detailed description; social synchronizationis as described in elsewhere in this detailed description; and socialsecurity is as described elsewhere in this detailed description.

In another embodiment (4) , the invention provides the apparatus of (1), wherein the instruction set includes one or more of the Dartinstructions selected from the set consisting of BUILTIN_INSTRUCTION,OEM_BUILTIN_INSTRUCTION, PROFILE_INSTRUCTION, and SAVE_INSTRUCTION.

In another embodiment (5) , the invention provides the apparatus of (1), wherein the Interoperability Instruction Set is the DartInstructionSetand is carried out by an Interoperability Engine.

In another embodiment (6) , the invention provides the apparatus of (5), wherein the Interoperability Engine is a DartEngine.

In another embodiment (7) , the invention provides the apparatus of (1), wherein the instruction set is embodied in one or more of: (1) asoftware product running on a processor with a different nativeinstruction set, whether the native instruction set is embodied inhardware, software, firmware, microcode, hardware logic or anycombination thereof; (2) firmware, or microcode coordinating anddirecting the activities of hardware logic units; (3) hardware logic; or(4) any combination of these.

In another embodiment (8) , the invention provides the apparatus in (1), further comprising: common user interface (UI) interoperability meansfor manipulating text, symbolic information, and images.

In another embodiment (9) , the invention provides the apparatus in (8), wherein: the common user interface interoperability means comprisecommon user interface interoperability instructions; and the userinterface interoperability instructions include instructions for atleast one of decoding, encoding, compressing and decompressing andmanipulating and rendering pictures, bitmaps, sounds, input events,text, symbols, audio/video, or other digitally encoded entity, and anycombination of one or more of these.

In another embodiment (10) , the invention provides the apparatus in (1), further comprising: a processor or CPU, memory coupled to theprocessor or CPU, and an input/output (I/O) interface being operable tosupport communications with the processor by an entity external to theapparatus.

In another embodiment (11) , the invention provides the apparatus in (1), further comprising: an operating environment associated with theprocessor, memory, and input/output interface for performing memoryaccess, computation, test and branch, and input/output instructions atleast for carrying out general purpose computing tasks.

In another embodiment (12) , the invention provides the apparatus in (1), further comprising: device performance enhancing interoperabilitymeans used to extend the practical reach of common binary applicationsand renditions to lower performance devices because the instructions areimplemented and executed in the native code format of a physicalprocessor or CPU of the device as part of the engine, rather then by asequence of slower executing emulated instructions of the applicationprogram used for binary compatibility.

In another embodiment (13) , the invention provides the apparatus of(12) , wherein the performance enhancing interoperability means(instructions) include instructions for one or more of the following:CPU intensive cryptographic operations on large numbers including one ormore of multiplication, division, addition, subtraction, exponentiation,modular exponentiation, hashing, random number generation, digitalsignature generation and verification, key pair generation, the encodingand decoding of data to be secured or read, and any instruction or setof instructions performing a combination of any two or more of these.

In another embodiment (14) , the invention provides the apparatus of(13) , wherein the processor or CPU intensive cryptographic operationson large numbers include operations involving individual numbers thatmust be stored in multiple memory words to assure that the values can beaccurately and precisely represented and or to ensure a proper degree ofsecurity.

In another embodiment (15) , the invention provides the apparatus of(12) , wherein the performance enhancing interoperability means are forperforming one or more of the following CPU intensive graphicsoperations: bitmap copying, bitmap scaling, bitmap stretching, bitmaptransposing, bitmap blending, bitmap filling, curve generation, linegeneration, circle generation, polygon rendering, piecewise linear curvegeneration or rendering, hit detection, font character generation andplacement, and any combination of these.

In another embodiment (16) , the invention provides the apparatus of(12) , wherein the performance enhancing interoperability means(instructions) are for performing one or more of the following CPUintensive text or symbol processing operations: XML parsing, textsearching, text insertion, text deletion, text database operations,text-to-text representation conversions, text manipulation,text-to-symbol manipulation, symbol-to-symbol manipulation, and anycombination of these.

In another embodiment (17) , the invention provides the apparatus of(12) , wherein the performance enhancing interoperability means are forperforming one or more of the following CPU intensive media processingoperations: audio decompression, video decompression, picturedecompression, dataset decompression, audio compression, videocompression, picture compression, dataset compression, digital imageprocessing, digital audio processing, dataset processing, databaseoperations and any combination of these.

In another embodiment (18) , the invention provides the apparatus in (1), further comprising: capabilities exposing interoperability means forexposing any characteristics and functions, including any uniquecapabilities and functions, of a particular device to software and/orfirmware applications and other devices that are being teamed or havebeen teamed through the use of the recruitment procedure or methodology.

In another embodiment (19) , the invention provides the apparatus in (1), further comprising: security maintenance interoperability instructionsfor accessing and setting or resetting of: security features, groupingof devices with a particular set of cross device access rights, andaccess rights for applications to devices and resources.

In another embodiment (20) , the invention provides the apparatus in (1), further comprising: containment interoperability means allowseparately generated Darts or executable procedures, to dynamicallybecome part of the linear tasking based runtime environment of otherDarts or executable procedures, whether as a child that inherits itsenvironment from its parent or as a parent which provides its runtimeenvironment to any child, or where the generated Dart or executableprocedure serve as both parent and child of other executable procedures.

In another embodiment (21) , the invention provides the apparatus in (1), further comprising: containment (DartContainment) interoperabilityinstructions to allow separately generated Darts, whether generated byone or more DartTools or by Dart Creationism, to dynamically become partof the Linear Tasking based DartRuntime of other Darts, whether as achild Dart that inherits its environment from its parent Dart or as aparent Dart which provides its runtime environment to any child Darts,or where the generated Darts serve as both parent and child of otherDarts.

In another embodiment (22) , the invention provides the apparatus in (1), wherein: Darts effectively extend the execution of an originating Dartacross other separately generated Darts that are collected, maintainedand/or run as part of the operation of the originating Dart.

In another embodiment (23) , the invention provides the apparatus in (1), further comprising: storage interoperability instruction means foraccessing digital data storage devices and optionally including any oneor more of hard disk drives, battery backed up memory, flash storagedevices, or other devices which can preserve data while the main poweris removed or turned off on the device.

In another embodiment (24) , the invention provides the apparatus in (1), further comprising: compatibility interoperability means(instructions) for signaling, retiring and otherwise managing DartEventsor events and an event queue (EventQueue) or DartEventQueue which drivesasynchronous and or synchronous processing of a Dart executing acrossone or more devices.

In another embodiment (25) , the invention provides the apparatus in (1), further comprising: unique capabilities instructions to expose anycharacteristics and functions of a particular device to softwareapplications and other devices.

In another embodiment (26) , the invention provides the apparatus in(25) , wherein the unique capabilities instructions may include aninstruction selected from the set of instructions consisting of: a DartPROFILE_INSTRUCTION that takes one or more IDs for a resource orcapability and returns a value, structured values, or a list ofstructured values.

In another embodiment (27) , the invention provides the apparatus in(26) , wherein the ID may be just a scalar value pre-assigned toindicate a particular resource or capability, or a major scalar valueindicating the general category plus a scalar minor value, or the ID maycontain a plurality of scalar values containing any combination of amanufacturer id, major category and minor category.

In another embodiment (28) , the invention provides the apparatus in(27) , wherein one of the major scalar values is a manufacturer IDscalar value that is assigned with a single unique value for everydifferent manufacturer and used to separate the ID values so thosespecific to a manufacturer can not conflict with those specific to adifferent other manufacturers' IDs for resources or capabilities.

In another embodiment (29) , the invention provides the apparatus in(25) , wherein the unique capabilities instructions include an originalequipment manufacturer OEM instruction which takes as a parameter amanufacturer ID scalar value, a parameter block specifying the operationto be performed and all the parameters needed to carry out a particularunique application program interface to the unique capabilities.

In another embodiment (30) , the invention provides the apparatus in(25) , wherein the original equipment manufacturer (OEM) instructioncomprises a Dart OEM _BUILTIN_INSTRUCTION instruction.

In another embodiment (31) , the invention provides the apparatus in(27) , wherein any other instructions that are part of any non-Dartinteroperability instruction set is used to expose unique resources andcapabilities of a device through a manufacturer specified applicationprogram interface.

In another embodiment (32) , the invention provides the apparatus in(31) , further comprising other instructions that are part of anynon-Dart interoperability instruction set performs and or is used toexpose unique resources and capabilities of a device, that operateanalogously to the Dart PROFILE_INSTRUCTION and the DartOEM_BUILTIN_INSTRUCTION instruction.

In another embodiment (33) , the invention provides the apparatus in(31) , further comprising an instruction set creating a commonprocedural environment across homogeneous and heterogeneous devices, theinstruction set comprising: a plurality of instructions designed andoptimized to perform all necessary interoperability operations betweenand among any of a plurality of homogeneous and heterogeneous devices;and the instructions being dispatched to functions which are compiled orassembled into the native code of the processor of the destinationdevice.

In another embodiment (34) , the invention provides an interoperabilityinstruction set creating a common procedural environment acrosshomogeneous and heterogeneous devices, the instruction set comprising: aplurality of instructions designed and optimized to perform allnecessary interoperability operations between and among any of aplurality of homogeneous and heterogeneous devices; and the instructionsbeing dispatched to functions which are compiled or assembled into thenative code of the processor of the destination device.

In another embodiment (35) , the invention provides an interoperabilityinstruction set as in (34), wherein the interoperability instruction setcomprises the Dart instruction set as implemented in the DartEngine of aDartPlayer running on one or more DartDevices.

In another embodiment (36) , the invention provides an interoperabilityinstruction set as in (34), wherein the interoperability instruction setexecutes in a portable engine creating a common procedural environmentin the devices for executing at least one of a device resource andcapability recruitment application and another application unmodified toexecute on the dissimilar heterogeneous device.

In another embodiment (37) , the invention provides an interoperabilityinstruction set as in (34), wherein the interoperability instruction setincludes at least one and any combination of the following instructions:recruitment instructions, profile instructions, synchronizinginstructions, user interface (ui) and graphics instructions, powermanagement instructions, connection and session management instructions,storage instructions, rendition instructions, creationism instructions,and application parts management instructions.

In another embodiment (38) , the invention provides an interoperabilityinstruction set as in (34), wherein the addressing field or fields ofthe instruction can reference at least: (1) a registers address space,whether or not disjoint from the other address spaces; (2) a main dataaddress space, whether or not disjoint from the other address spaces;(3) a stack address space, whether or not disjoint from the otheraddress spaces; and (4) an application heap address space, whether ornot disjoint from the other address spaces.

In another embodiment (39) , the invention provides an interoperabilityinstruction set as in (38) , wherein the addressing field or fields ofthe instruction can further reference: (5) one or more of the followingadditional address spaces in any combination: (i) one or more disjointvirtual pointer address spaces; (ii) a separate application heap elementaddress space; (iii) a virtualized local address space to the currentfunction data space which is a contiguous subset of another addressspace; and (iv) one or more virtualized object instance address spaceswhich are contiguous subsets of other address spaces.

In another embodiment (40) , the invention provides an interoperabilityinstruction set as in (38), wherein all address spaces are addressed byan N bit number, and a number of bits, M, of the N bits specify which ofdifferent possible 2 to the M power of address spaces the N-M remainingbits is actually referencing.

In another embodiment (41) , the invention provides an interoperabilityinstruction set as in (39), wherein all address spaces are addressed byan N bit number, and a number of bits, M, of the N bits specify which ofdifferent possible 2 to the M power of address spaces the N-M remainingbits is actually referencing.

In another embodiment (42) , the invention provides an interoperabilityinstruction set as in (40), where the address space is actually in unitsof 2 to the power of M of the native processor's smallest directlyaccessible words so that using the N-M address space can still specifythe entire direct address space of the native processor, only withoutthe ability to directly address units of memory smaller then 2 to thepower of M.

In another embodiment (43) , the invention provides an interoperabilityinstruction set as in (38), wherein virtual address pointer addressspaces are used.

In another embodiment (44) , the invention provides an interoperabilityinstruction set as in (38), where the decoding of the address fields ofinstructions for data or code in any or all of the address spaces ischecked by the processor decoding the instructions to ensure that noaccess will take place outside of the currently restricted bounds of theunderlying memory, or other hardware accessible through the use of thedecoded address fields.

In another embodiment (45) , the invention provides the apparatus of (1), wherein the instruction set includes instructions used to access to afile system that supports traditional hard files and one or more of thefollowing types of files in any combination: (i) a memory file; (ii) asubfile file; (iii) a part subfile file; (iv) a part descriptoroverriding file; and (v) any combination or sequential or non-sequentiallayering of these types of files.

In another embodiment (46) , the invention provides the apparatus of(45) , wherein the memory file is a virtualized a traditional hardfileaccessible through the same methods for accessing a hardfile but wherethe data is kept in changeable main memory of a processor rather than ontraditional hard storage such as a hard disk drive or other physicaldevice other than the type commonly used for main memory access ofprocessors.

In another embodiment (47) , the invention provides the apparatus of(45) , wherein the subfile file is a virtualized file representing alinear contiguous range of data inside another file, and where thelinear contiguous range is virtualized as beginning at a logical offsetof zero, before which there is no data, and the end of the range isbounded by the offset representing the length of the range of data, atand beyond which there is no data.

In another embodiment (48) , the invention provides the apparatus of(45) , wherein the part subfile file is a subfile which represents alogical individually addressable part inside a file holding anapplication package of one or more independently executable images.

In another embodiment (49) , the invention provides the apparatus of(45) , wherein the part descriptor overriding file), where it existswith the same identifier value as does a specific part, will serve tooverride any or existing data which once represented the part, whetherthe existing data is inside a file holding an application package of oneor more independently executable images or it is inside any otherseparate file.

In another embodiment (50) , the invention provides the apparatus of(49) , wherein the part descriptor overriding file is used to logicallyreplace old data for a part without the need to change the existing datainside other files.

In another embodiment (51) , the invention provides the apparatus of(49) , wherein the part descriptor overriding file can represent thedata for a part as being one of: logically deleted or non-existent;physically stored elsewhere or in another file as identified in thedescriptor overriding file; representative of the access rights orallowed usage of the part data; and represented inside the descriptorfile itself.

In another embodiment (52) , the invention provides a method foreffecting an interoperability instruction set (IIS) in an apparatus, themethod comprising: providing a processor and memory coupled to theprocessor, and an input/output (I/O) interface to support communicationswith the processor by an entity external to the apparatus; supportingexecution interoperability methodologies involving at least one of arecruitment procedure, a renditioning procedure, a creationismprocedure, a vertical layering procedure, a linear-tasking procedure, asocial synchronization procedure, and a social security procedure; andproviding communications interoperability instructions for opening andmaintaining a communication session and the procedures, data, contentand/or other information that may be exchanged between communicatingdevices during a communication session.

In another embodiment (53) , the invention provides a method as in (52), wherein: the execution interoperability methodology comprises at leastone interoperability instruction for carrying-out the interoperabilitymethodology.

In another embodiment (55) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance having a processor and memorycoupled to the processor and an input/output (I/O) interface to supportcommunications with the processor by an entity external to theapparatus, to function in a specified manner for effecting aninteroperability instruction set (IIS) in an apparatus, the programmodule including instructions for: supporting execution interoperabilitymethodologies involving at least one of a recruitment procedure, arenditioning procedure, a creationism procedure, a vertical layeringprocedure, a linear-tasking procedure, a social synchronizationprocedure, and a social security procedure; and providing communicationsinteroperability instructions for opening and maintaining acommunication session and the procedures, data, content and/or otherinformation that may be exchanged between communicating devices during acommunication session.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XIV. Creationism

In another aspect of the invention, system, apparatus, method andcomputer program for Creationism are provided. Creationism includes amethodology which enables an interoperability application to createother interoperability applications, which in turn can create still moreinteroperability applications. This allows for the efficient dynamicgeneration and distribution of application, data and content indiffering forms across a world of connected and intermittently connecteddevices.

In the one advantageous embodiment and implementation, this is theability for Darts to create other Darts (FIG. 12 700) or Dart procedures(DartProcedures FIG. 14 4000) which can then create still other Darts orDartProcedures. Creationism is embodied throughout the Dart Platform(DartPlatform FIG. 3). For example, the Dart tools (DartTools FIG. 3200) allow for the specification of Parts in the source code, andcompile the source code into a DartMaster (FIG. 12 230) containing theseParts. The DartMaster when played on a MasterPlayer (FIG. 12 203) cancollect more resources if necessary and provide the user interface forrequesting any needed information on which Renditions and parts toinclude in the output Dart or DartProcedure.

Any Dart running on any Dart Player can include instructions from theDartInstructionSet supported in the DartEngine to dynamically formRenditions from Parts and package them together efficiently intoDartFormat files. This process of Dart creation of customized Darts cango on indefinitely so long as the created Darts and Procedures areformed with the data, content and procedures necessary to do so.

Creationism provides a method, associated procedures and computerprogram code and product for enabling an interoperability application tocreate other interoperability applications, which in turn can createstill more interoperability applications in a recursive and or serialand or fanout manner. Tools are provided for compiling source code (FIG.3 100) and optionally source data and/or source resources into a binaryimage containing at least one Rendition. It also uses and may provideexecutable instructions and/or application programming interface(s) fordynamically assembling digital parts into a binary image containing aset of Renditions (e.g. Dart Renditions) and parts that control how theRenditions are to be assembled based on the communications capabilities,device characteristics and environment of a target device.

The tools may include Dart Tools (FIG. 12 200). The binary image mayinclude or consist of a Dart in the Dart format (FIG. 3 300, FIG. 13,FIG. 14), or may be a differently formatted binary image or an imageencoded in a non-binary form. The digital parts may be procedures, datasets, and/or content of any type or form expressed as a structuredsequence of numbers.

Additional particular embodiments of creationism are now set forth. Inone embodiment (1) , the invention provides a method for enabling aninitial individually executable image or package of individuallyexecutable images to dynamically generate at least one other targetindividually executable data image or package of individually executabledata images to carry out the intent of the initial executable image orpackage of individually executable images, the method comprising: (1)collecting first information about at least one of the characteristics,content, resources, or capabilities of devices and or other environmentsfor execution of generated executable images or packages of images whichmight be of use in carrying out the intent or part of the intent of thegenerating executable image or package; (2) determining how to assembleat least one of: (i) parts of its own image, (ii) collected information,or (iii) programmatically generated information, to make efficient useof the resources, capabilities and content of the target devices orenvironments for which information was collected; and (3) gatheringsecond information necessary to generate one or more other independentlygenerated executable data images or image packages as needed to carryout the intent of the generated target executable image or image packagein an unlimited sequence.

In another embodiment (2) , the invention provides a method as in (1) ,further comprising: (4) using the gathered information to generate theone or more independently generated executable data images or imagepackage.

In another embodiment (3) , the invention provides a method as in (1) ,wherein one or more of the following are true in any combination: (1)the individually executable image is a rendition or a Dart Rendition;(2) the package of individually executable data images conforms to aninteroperability format or to the DartFormat; (3) the generation iscarried out as part of a device teaming recruitment method; (4) thegathering information necessary is collected or formed into a singleDart RenditionsTable which references one or more collected or formedDart RenditionTables which in turn reference one or more collected orformed DartParts, to be located using a single collected or formed DartPart Table, which is in turn located using a Dart Trailer or any otherof the methods described in an interoperability format; (5) thecollecting information is carried out at least in part by use ofprocedures sent to execute on the target device or devices; (6) thecollecting information is carried out at least in part by use ofDartProcedures sent to execute on the target device or devices; (7) thecollecting information is carried out at least in part by executing thePROFILE_INSTRUCTION of the DartInstructionSet on the target device ordevices; and (8) the using the gathered information to generate the oneor more independently generated executable data images or image packagesis carried out at least in part by the DartInteroperablityInstructionSetSAVE_INSTRUCTION instruction and or the BUILTIN_INSTRUCTION instructionbeing executed on the target device or devices by an interoperabilityengine or the DartEngine.

In another embodiment (4) , the invention provides a method as in (1) ,further comprising by forming and storing the generated executableimages or packages on physical media.

In another embodiment (5) , the invention provides a method as in (1) ,wherein the creationism method is used for one or more of the followingor any combination thereof: (1) distributing an application and orapplication package of individually executable images and or dataset ordatasets of any kind to one or more devices; (2) customizing anapplication and or application package of individually executable imagesand or dataset or datasets; (3) creating a one time use executable imageor package of executable images to carry out the intent of theinitiating executable image on one or more target devices; and (4)creating an executable image or package of executable images whichcontains a selected subset of content and or resources and or data andor code that is part of or is collected by the initiating image.

In another embodiment (6) , the invention provides a method as in (5) ,wherein the customizing an application and or application package ofindividually executable images and or dataset or datasets according toone or more of the following list: (a) the needs of the target device ordevices; (b) the environment of the target device or devices; (c)limitations of the target device or devices; (d) capabilities of thetarget device or devices; and (e) access by the target device ordevices, whether current or future, to any other devices according toany of the items in this list.

In another embodiment (7) , the invention provides a method as in (5) ,wherein the creating an executable image or package of executable imageswhich contains a selected subset of content and or resources and or dataand or code that is part of or is collected by the initiating image sothat the image or package is any one or more of or any combination of:(a) small enough for efficient storage on the target device or devices;(b) small enough to be transported more quickly; and (c) optimized toonly hold the subset of interest or a subset which conforms to a givencriteria for inclusion and or collection of related items and or anintended purpose or purposes.

In another embodiment (8) , the invention provides a method as in (1) ,further including generating an executable image or image package thatcarries out a distribution of the generated executables or packages toother environments, storage devices, and/or devices for which theparticular generated executables or packages where generated andintended to be distributed.

In another embodiment (9) , the invention provides a method as in (1) ,further including initiating the execution of the generated executableimages or image packages on other devices as needed to carry out theintent of the generating executable images or package.

In another embodiment (10) , the invention provides a method as in (9) ,wherein the initiation of execution is one of a command-basedinitiation, a time-based initiation, a schedule-based initiation, astatistically-based initiation, an event-based initiation, or any othercriteria based initiation of execution.

In another embodiment (11) , the invention provides a method as in (10), wherein the initiation is controlled or influenced by the DartRuntime.

In another embodiment (12) , the invention provides a method as in (1) ,wherein the intent or part of the intent of the executable image orpackage that is generating other executable data images or packages isto spread the executable image or image package, or parts of theexecutable image or image package to one or more devices and/or storagemediums.

In another embodiment (13) , the invention provides a method as in (4) ,wherein the executable images or packages are spread using a deviceRecruitment procedure and using an interoperability instruction set.

In another embodiment (15) , the invention provides a method as in (13), wherein the recruitment method comprises the Dart recruitmentprocedure and using the DartInstructionSet.

In another embodiment (16) , the invention provides a method as in (1) ,wherein the package of individually executable images is a tightlyintegrated package of interrelated individually executable images.

In another embodiment (17) , the invention provides a method as in (1) ,wherein the package of executable images comprise a package of tightlyintegrated individually executable data images.

In another embodiment (18) , the invention provides a method as in (1) ,wherein the method further includes this process of generation repeats aplurality of times.

In another embodiment (19) , the invention provides a method as in (18), wherein the plurality of times may be any number of times), whereinsuch any number of times may be an indefinite number or times orindefinitely.

In another embodiment (20) , the invention provides a method as in (1) ,wherein the method further includes repeating the process of generationa plurality of times recursively in a chain of executable images andpackages of executable images.

In another embodiment (21) , the invention provides a method as in (1) ,wherein the intent is determined at least in part by the applicationdesigner's and/or implementers' intended purpose and functions asembodied in the application.

In another embodiment (22) , the invention provides a method as in (1) ,wherein the intent is determined at least in part by a separateapplication generation program or set of such programs intended purposeor functions embodied in the application.

In another embodiment (23) , the invention provides a method as in (1) ,wherein collection is accomplished using Recruitment carried out by thesending of procedures which may or may not be DartProcedures and/orDarts.

In another embodiment (24) , the invention provides a method as in (1) ,wherein the information is collected from storage or other programsrunning on the device on which the generating executable image orpackage is running.

In another embodiment (25) , the invention provides a method as in (1) ,wherein the information is collected from storage or other programsrunning on devices other than the device on which the generatingexecutable image or package is running.

In another embodiment (26) , the invention provides a method as in (1) ,wherein the information is collected over any communication mediumand/or protocol whether wired or wireless or through the physicaltransport of storage or computing devices between the generating deviceand any other devices.

In another embodiment (27) , the invention provides a method as in (1) ,wherein the determining is a determining accomplished by procedures thatare part of the generating executable image.

In another embodiment (28) , the invention provides a method as in (1) ,wherein efficient use of resources is determined by the execution ofprocedures that are part of the originating application designed by theapplications designers and/or implementers to carry out the intent.

In another embodiment (28) , the invention provides a method as in (28), wherein the collection is accomplished at least in part through theuse of at least one of Recruitment as described elsewhere in thisdetailed description and DartProcedures.

In another embodiment (30) , the invention provides a method as in (1) ,wherein the gathering uses at least any one of an external executabletool, an internal procedure, an instruction, and/or a system call by thegenerating individually executable image or image package.

In another embodiment (31) , the invention provides a method as in (1) ,wherein the generating of an executable image or image package isaccomplished at least in part through the use of DartEvents to carry theinformation and do the distribution to DartPlayers.

In another embodiment (32) , the invention provides a method as in (1) ,wherein the intent or a part of the intent of the executable image orpackage that is generating other executable data images or packages ofimages in a possibly unending chain is to perform synchronization ofdata or cooperative programs across any number of devices in a mannerwhere the data, content, and/or procedures associated or collected byany of the set of generated executable images or packages is tocooperatively synchronize, merge, manage, transcode, and/or collectprograms, content, executable images, packages of executable images,pictures, or any other program, database, or content expressible as anordered collection of digital data.

In another embodiment (33) , the invention provides a method as in (1) ,wherein, the intent or part of the intent of the executable image orpackage that is generating other executable data images or packages isto include only those parts of the data, content, procedures of thegenerating individually executable images or packages, collected datacontent and procedures, and/or programmatically generated content,procedures or collected data, that are needed to carry out a particularpart of the intent of the originating executable images or packages thatare to be carried out using the generated images or packages on acomputational device or set of devices other than the one that theoriginating executing image or package is running on at the time thegeneration occurs.

In another embodiment (34) , the invention provides a method as in (33), wherein each image or image package generated for execution on eachdevice is intelligently custom built in a form conforming to oroptimized for the characteristics, capabilities and/or content of eachdevice or set of devices that each generated image is intended for.

In another embodiment (35) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for enabling an initial individually executable image or packageof individually executable images to dynamically generate at least oneother target individually executable data image or package ofindividually executable data images to carry out the intent of theinitial executable image or package of individually executable images,the program module including instructions for: (1) collecting firstinformation about at least one of the characteristics, content,resources, or capabilities of devices and or other environments forexecution of generated executable images or packages of images whichmight be of use in carrying out the intent or part of the intent of thegenerating executable image or package; (2) determining how to assembleat least one of: (i) parts of its own image, (ii) collected information,or (iii) programmatically generated information, to make efficient useof the resources, capabilities and content of the target devices orenvironments for which information was collected; (3) gathering secondinformation necessary to generate one or more other independentlygenerated executable data images or image packages as needed to carryout the intent of the generated target executable image or image packagein an unlimited sequence; and (4) using the gathered information togenerate the one or more independently generated executable data imagesor image package.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XV. Interoperability Engine/DartEngine

Recall that the DartEngine is or includes software and or hardware usedto execute the instructions of Darts on a device and carry out theirintended purpose. The DartEngine and the device specific DartPlayer, inwhich it is encapsulated, provides the common execution and DartRuntimeenvironment which allows Recruitment and Renditioning to establishefficient teams of devices and spread their code, data and content asbest to carry out the intended purpose of Darts.

Additional particular embodiments of Interoperability Engine and a morespecific embodiment of an Interoperability Engine, the DartEngine, arenow set forth. In one embodiment (1) , the invention provides aninteroperability engine which enables or assists devices to interoperatewith each other, the engine comprising: (1) means for loading, running,and carrying-out at least part of the intent of an interoperabilitysoftware package having code and wherein at least part of the code isembedded in a sequence of instructions conforming to an interoperabilityinstruction set; (2) means for discovering other interoperabilitydevices; and (3) means for direct or indirect two-way communicationswith other interoperability devices.

In another embodiment (2) , the invention provides an interoperabilityengine as in (1) , wherein one or more of the following are true in anycombination: (1) the engine comprises a DartEngine; (2) theinteroperability software package comprises an interoperability softwarepackage or comprises a Dart conforming to the DartFormat; and (3) themeans to discover other interoperability devices is at least partiallycarried out using a recruitment procedure or a Dart recruitmentprocedure.

In another embodiment (3) , the invention provides an interoperabilityengine as in (1) , wherein the engine includes an event queue andcarries out instructions to support the use of the event queue byinteroperability application packages.

In another embodiment (4) , the invention provides an interoperabilityengine as in (1) , wherein the engine includes a computer programproduct for execution in a processor or logic circuit of the a devicecontaining the engine.

In another embodiment (5) , the invention provides an interoperabilityengine as in (1) , wherein the engine includes a hardware processorimplementing the engine within a device containing the engine.

In another embodiment (6) , the invention provides an interoperabilityengine as in (1) , wherein the engine includes a hardware processorimplementing a portion of the engine within a device containing theengine and a computer program product for execution in the hardwareprocessor of the a device containing the engine.

In another embodiment (7) , the invention provides an interoperabilityengine as in (6) , wherein a source code for the computer program andcomputer program product portion of the engine are segmented into aportable section which can be used without modification on allinteroperable devices and a hardware abstraction layer which may need todiffer for each different type of device.

In another embodiment (8) , the invention provides an interoperabilityengine as in (7) , wherein the hardware abstraction layer includes codeproviding access to one or more of the following device functions in anycombination: (a) memory allocation functions; (b) sound renderingfunctions; (c) power management functions; (d) printing functions; (e)display functions; (f) media rendering and or playback and ortranscoding functions; (g) computation functions; (h) storage functions;(i) communications functions; (j) device and or service discoveryfunctions; (k) profile information gathering functions about thecapabilities, resources, content and or environment of the device; (l)device configuration functions; (m) device status functions; and (n) anyother functions or operations that can be performed by the native devicehardware and software that can be exposed.

In another embodiment (9) , the invention provides a method foroperating an interoperability engine which enables or assists aplurality of devices to interoperate with each other, the methodcomprising: loading, running, and carrying-out at least part of theintent of an interoperability software package having code and whereinat least part of the code is embedded in a sequence of instructionsconforming to an interoperability instruction set; discovering otherinteroperability devices; and directly or indirectly communicating withother interoperability devices.

In another embodiment (10) , the invention provides a method as in (9) ,wherein one or more of the following are true in any combination: (1)the engine comprises a DartEngine; (2) the interoperability softwarepackage comprises an interoperability software package or comprises aDart conforming to the DartFormat; and (3) discovering of otherinteroperability devices is at least partially carried out using arecruitment procedure or a Dart recruitment procedure.

In another embodiment (11) , the invention provides a method as in (9) ,wherein the method further comprises: generating an event queue withinthe engine and executing computer code instructions to support the useof the event queue by the interoperability application package.

In another embodiment (12) , the invention provides a method as in (9) ,wherein the method further comprises: segmenting a source code into aportable section which can be used without modification on allinteroperable devices and a hardware abstraction layer which may need todiffer for each different type of device.

In another embodiment (13) , the invention provides a method as in (12), wherein the hardware abstraction layer includes code providing accessto and optionally executing one or more of the following devicefunctions and/or procedures alone or in any combination: (a) memoryallocation functions and procedures; (b) sound rendering functions andprocedures; (c) power management functions and procedures; (d) printingfunctions and procedures; (e) display functions and procedures; (f)media rendering and or playback and or transcoding functions andprocedures; (g) computation functions and procedures; (h) storagefunctions and procedures; (i) communications functions and procedures;(j) device and or service discovery functions and procedures; (k)profile information gathering functions and procedures about thecapabilities, resources, content and or environment of the device; (l)device configuration functions and procedures; (m) device statusfunctions and procedures; and (n) any other functions and procedures oroperations that can be performed by a native device hardware andsoftware that can be exposed.

In another embodiment (14) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for operating an interoperability engine which enables or assistsa plurality of devices to interoperate with each other, the programmodule including instructions for: loading, running, and carrying-out atleast part of the intent of an interoperability software package havingcode and wherein at least part of the code is embedded in a sequence ofinstructions conforming to an interoperability instruction set;discovering other interoperability devices; and directly or indirectlycommunicating with other interoperability devices.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XVI. Interoperability Device Enabling

Recall that Interoperability Device Enabling is the process of turning aconventional device into a highly interoperable DartDevice through theporting of a DartEngine as part of a DartPlayer. In addition,implementation of the Hardware Abstraction Layer needed to access thedevice specific information, capabilities and content of the device isalso required. At least one communications protocol must be implementedbefore a device with a DartPlayer becomes a DartDevice.

Additional particular embodiments of Interoperability Device Enabling,are now set forth. In another embodiment (1) , the invention provides amethod for using common software source code for an interoperabilityengine to create interoperability software needed to make a device aninteroperability device, the method comprising: (1) creating aninteroperability engine object or instance; (2) creating a deviceHardware Abstraction Layer (HAL) object or instance; (3) identifying andfilling in the functionality of all the predefined required halBasemember functions of the halBase class or specification; and (4) creatinga device specific Player object that substantially continually runs theengine on a thread of execution.

In another embodiment (2) , the invention provides a method as in (1) ,wherein the creating a device specific Player object that substantiallycontinually runs the engine on a thread of execution runs the engine asfollows: (a) call the engine's initialization function; (b) call theengine's process function in a loop that ends if a returned valueindicates an unrecoverable error occurred or that the engine is to beclosed down; (c) call the engine's un-initialize function; and (d) stopthe thread of execution.

In another embodiment (3) , the invention provides a method as in (2) ,wherein the creating includes creating by inheriting from a halBaseobject.

In another embodiment (4) , the invention provides a method as in (2) ,wherein the returned value returned comprises a non-zero value returned.

In another embodiment (5) , the invention provides a method as in (2) ,wherein the loop that calls a player base process member functioncomprises a loop that calls a playerBase::Process( ) member function ina loop until a non-zero value is returned.

In another embodiment (6) , the invention provides a method as in (1) ,wherein the creating of a device specific playback object includescreating by inheriting from a playerBase object.

In another embodiment (7) , the invention provides a method as in (1) ,wherein the processing engine comprises a software processing engine.

In another embodiment (8) , the invention provides a method as in (1) ,wherein the processing engine comprises a firmware processing engine.

In another embodiment (9) , the invention provides a method as in (1) ,wherein the processing engine comprises a Dart software processingengine.

In another embodiment (10) , the invention provides a method as in (1) ,wherein the software includes C++ programming code instructions.

In another embodiment (11) , the invention provides a method as in (1) ,wherein the functions of the HAL object that are identified include atleast one function from the set of functions consisting of: allocating asingle consecutive block of memory of a given size, returning the timein milliseconds, moving a bitmap in one of an identified standardformat, compile time variants to the screen at a given location, avirtual function for getting profile characteristics of another physicaldevice on which the processing engine will execute.

In another embodiment (12) , the invention provides a method as in (1) ,further comprising communicating the device specific playback object tothe device.

In another embodiment (13) , the invention provides a method as in (12), wherein the device comprises at least one of: a computer, a personaldata assistant (PDA), a cellular or wireless telephone, a radio, aprinter, an electronic device, a music player, a media player, a camera,or any one combination of thereof.

In another embodiment (14) , the invention provides a method as in (2) ,wherein the hal functions include interface functions for interactingwith the device specific hardware, software, firmware, or content forone or more of the following purposes: (1) getting device specificcapabilities and/or settings and/or status for a given id value or setof values that identify specific profile words, profile structures, orthe enumeration of profile structures that are to be gotten; and (2)executing predefined asynchronous operations so that the execution ofother activities of the interoperability device running on the engine'stread of execution can continue to run while the asynchronous operationscontinue.

In another embodiment (15) , the invention provides a method as in (14), wherein one of the parameters to the interface function is a valueuniquely assigned to each different manufacturer and another parameteris a pointer to a parameter block so that each manufacturer canoptionally define whatever sub-interface functions and/or parametersrequired to expose any unique capabilities, status, content, oroperations not otherwise accessible through the other interfacefunctions.

In another embodiment (16) , the invention provides the method as in(15) , wherein the uniquely assigned values are used to ensure therewill be no conflicts with other manufacturer generated sub interfaces.

In another embodiment (17) , the invention provides the method as in(16) , wherein the manufacturer can control access to their owndeveloped sub-interfaces in one or more of the following manners: (1)publish their manufacturer id, parameter block specification and/orsub-interface function specifications so that any other manufacturer canmake use of the developed sub interface functions to develop compatibledevices and applications; (2) keep their manufacturer id and orparameter block specification and or sub-interface specifications astrade secrets; and (3) protect the parameters or functions using thecryptographic instruction implementations in the engine, a shared secretbased algorithm, or any other cryptographic means so that it will bedifficult or impossible for other manufactures to understand or make useof the manufacturer developed sub interface functions.

In another embodiment (18) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for using software source code common among potentiallyinteroperable devices for an interoperability engine to createinteroperability software needed to make a device an interoperabilitydevice, the program module including instructions for: (1) creating aninteroperability engine object or instance; (2) creating a deviceHardware Abstraction Layer (HAL) object or instance; (3) identifying andfilling in the functionality of all the predefined required halBasemember functions of the halBase class or specification; and (4) creatinga device specific Player object that substantially continually runs theengine on a thread of execution.

In another embodiment (19) , the invention provides a computer programproduct as in (18) , wherein the instruction for creating a devicespecific Player object that substantially continually runs the engine ona thread of execution runs the engine according to the steps of: (a)calling the engine's initialization function; (b) calling the engine'sprocess function in a loop that ends if a returned value indicates anunrecoverable error occurred or that the engine is to be closed down;(c) calling the engine's un-initialize function; and (d) stopping thethread of execution.

In another embodiment (20) , the invention provides a computer programproduct as in (18) , wherein the functions of the HAL object that areidentified include at least one function from the set of functionsconsisting of: allocating a single consecutive block of memory of agiven size, returning the time in milliseconds, moving a bitmap in oneof an identified standard format, compile time variants to the screen ata given location, a virtual function for getting profile characteristicsof another physical device on which the processing engine will execute.

In another embodiment (21) , the invention provides a computer programproduct as in (18) , wherein the hal functions include interfacefunctions for interacting with the device specific hardware, software,firmware, and/or content for one or more of the following purposes: (1)getting device specific capabilities and/or settings and/or status for agiven id value or set of values that identify specific profile words,profile structures, or the enumeration of profile structures that are tobe gotten; and (2) executing predefined asynchronous operations so thatthe execution of other activities of the interoperability device runningon the engine's tread of execution can continue to run while theasynchronous operations continue.

In another embodiment (22) , the invention provides a computer programproduct as in (19) , wherein one of the parameters to the interfacefunction is a value uniquely assigned to each different manufacturer andanother parameter is a pointer to a parameter block so that eachmanufacturer can optionally define whatever sub-interface functionsand/or parameters required to expose any unique capabilities, status,content, or operations not otherwise accessible through the otherinterface functions.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XVII. Interoperability Security Model/DartSecurity

Recall that DartSecurity is a system and method for providing theinfrastructure needed for protecting the integrity of a device and itscontent from malicious or accidental damage.

A robust security infrastructure is desirable to serve as a basis forprotecting a device, its content or facilities from being abused,corrupted or otherwise compromised or used in any undesired manner. Suchan infrastructure is the inventive DartSecurity system embodied in thepreferred manner by elements of the DartPlatform as shown in FIG. 29.

In the preferred embodiment, when a DartDevice executes the DartPlayerfor the first time, part of the initialization process of the DartEngineis to perform the following:

1. Gathering enough entropy suitable for generating public and privatecryptographic key pairs and unique ids (uids) to statistically guaranteethat the generated key pair and unique ids are truly unique from thosegenerated in a similar manner, but using different gathered entropy.

Entropy can be reliably gathered by DartDevices by digitally samplingaspects of the world outside of the device which are not synchronized inany way to the clock running the sampling hardware. Communicationsmechanisms used to talk to other devices are often good sources ofentropy because the exact timing of packets, timeouts and variations ofdata are affected by electromagnetic interference and quantum noiseuncorrelated to the clock of the processor controlling the sampling.Another easy source of entropy is for the device to ask for inputs froma human user and sample a fine grained clock whenever the user providesany input. The entropy containing timing and data samples can be hashedinto a set of data values used to seed a random number generator.Although it is difficult in practice to know when enough entropy hasbeen gathered, estimates can be made and over-sampling used to ensurethat enough has been gathered and maintained in the hashed data values.

2. Using a conventional random number generation algorithm andconventional cryptographic operations to generate a universally uniquepublic/private key pair and a universally unique DartDevice uid.used touniquely identify the device for all time. In a preferred implementationall uids are 128-bits long.

3. The private key and entropy hashed set of data values are stored onthe device in any conventional manner known in the state of the art forobscuring and or hiding data on a device, yet still having the use ofthe data by the engine practical.

4. Limiting access to resources of data, content or facilities of thedevice based on a set of binary flags. In a preferred implementation,these flags are 1 if access to the corresponding data for facility ofthe device is to be blocked. As examples, individual flags may exist forthe following resources:

-   -   a. Blocking access to data or content that is marked as private,    -   b. Blocking access to opening communications sessions to another        device,    -   c. Blocking access to the native storage devices, and    -   d. Blocking access to user interface elements such as the screen        and keyboard and mouse and sound generation.

5. Every Dart or DartProcedure that is to be loaded to run on theDartEngine has explicit or implicit sets of flag values controlling itsaccess to resources of the DartDevice. Every DartDevice has explicit orimplicit sets of access flags mapped to levels of security. If theapplication or the device sending the application does not have all theflags for the resources it needs to access set to zero on the targetdevice the Dart application or DartProcedure will not be loaded unlessthe device first gets the proper authorization from a user or otherdevice with acceptable credentials or flags authorizing the Dart orDartProcedure to run on the specific device. Access flags tied to theuids of Darts, DartProcedures, or DartDevices can be maintainedpermanently on a target DartDevice, or for a specific period of time, orfor the duration of the execution of the Dart or DartProcedure or theduration of the communications session between the devices.

6. If a Dart application attempts to cause the DartEngine to access anyof the resources on its behalf to which the application's context flagsare set to 1, the DartEngine will immediately stop executing theinstructions of the application and close the application down.

In one preferred implementation Darts must be digitally signed alongwith the flags sets for access. Encryption/Decryption of parts orcontent or of communications sessions between devices is also part ofthe DartSecurity system.

FIG. 25 shows a DartDevice 400 with some of the Security Systemelements. The DartEngine has instruction based and methods forsupporting encryption 150, entropy gathering 153, random numbergeneration 152, and securing all communications between devices usingthe DartSecure protocol to ensure that any communications between anytwo DartDevices can be automatically encrypted so that DartEngine basecommunications does not need to rely on any outside protocols to protectthe communication of data. The DartSecure protocol 153 can beimplemented using the Secure Sockets Layer (SSL) protocol or any otherprotocol which establishes a secure communications link between devices.In a preferred implementation a subset of the functionality in the SSLprotocol standard is implemented to reduce the size and complexity asmany of the options in SSL are not needed. All the necessary largenumber math cryptographic primitives 150 and 151 are shown as part ofthe DartEngine. For performance reasons it is high desirable to have thecryptographic math primitives implemented in the native code of theengine instead of inside the Dart.

FIG. 25 Also shows that the DartEngine maintains lists of uids 155 and156 corresponding to levels of security. The figure also shows thatthere is a table that maps level numbers to sets of access rights flags159. The list of device and application ids that are part of aparticular level of rights is maintained by the DartEngine and in apreferred mode of operation is allowed to propagate transitively whenpermission is allowed to do so from device to device so that any devicein the list can gain access to the new device without the need forfarther collection of credentials. This inventive transitive teaming ofdevices and applications at specific levels of security, SocialSecurity,is further described elsewhere in this document.

To ensure that even after presenting its flags, passing muster, andgetting loaded, the Dart application or DartProcedure cannot purposelyor accidentally access resources that are off limits; all such attemptedaccesses are checked by the engine using a context data structure 160and 170 which carries with it all the permission flags originallypresented by the Dart or DartProcedure.

Note that a revocation list 155 can be transitively maintained asdescribed further elsewhere in this document as, SocialSynchronization,so that any device previously teamed with other devices at a particularsecurity level can have its access rights logically revoked forinteroperability with any device which transitively learns of therevocation from any other device of the team. In one preferredimplementation if a revoked device comes in contact with a teamed devicewith knowledge of the revocation, the DartEngines will render therevoked device inoperable or partially inoperable. Revocation of accessrights across a team of interoperable devices with common access rightsis not perfect but in many situations it is a practical system forcarrying out the revocation of the access rights of a lost, misbehavingor malevolent device across a team of possible intermittently connecteddevices.

The method of spreading of access rights or the revocation thereof ismuch like humans who tell their friends about an abusive person, and theinformation spreads like gossip from person to person so that mostpeople know about and can stay away from the abusive person even if theyhave never talked directly to the person with the first knowledge of theabusiveness of the person.

The DartEngine context in addition to access flags also containsinformation that the engine uses to limit access by the running Dart orDartProcedure to any memory or storage or code that is not a part of therunning Dart. This limiting of access is known in the state of the artas a “sandbox.” The DartEngine enforces such a sandbox on all Darts andDartProcedures so that Darts cannot be used to access the device outsideof the functions, memory and storage portions explicitly provided foruse by Darts and DartProcedures running in the sandbox.

Darts

In one preferred implementation of the invention there is littlespecialization between what is conventionally though of as code, data,operating system, graphics or user input subsystems, and threading, orfor that matter between content, programs, or even devices. TheDartPlatform (FIG. 3) can intelligently mix and match softwareprocedures, code, data, content, and device electronics into anintegrated virtual system which carries out the intent of an applicationas designed by the Dart designer and then specified in the DartSource(FIG. 3 100).

Darts built by the DartTools can self-optimize, reorganize, send andsynchronize versions or extensions of itself for the efficient sharingof programs, controls, hardware resources and content between and amongvarious devices.

Darts are built with all the data, content and procedures needed toestablish an integrated working application capable of running well inany number of dissimilar devices, but also across numerous similar ordissimilar devices simultaneously.

In addition, the runtime environment of a single Dart can alsodynamically or statically include other Darts as parent Dart or childDart extensions of any Dart. Thus the DartRuntime environment is uniquein its reach across unknown devices and unknown separately producedDarts.

Unlike interoperability technologies employed currently, Dart technologythough not limited to such tasks, is optimized around human sized dataand tasks such as picture slide shows, appointment calendars, contacts,control panels and messages. The response time of the system is fastenough to do motion video and respond to button presses or requests tofind a particular contact in a personal contact list within a fifth of asecond. For humans, such response times are nearly indistinguishablefrom being instantaneous. Dart response times requirements are much morelenient compared to most real-time operating systems at the base of mostapplication environments, because real-time operating systems aregenerally expected to run applications such as data servers which musthandle hundreds of operations per second. Because Darts only need to beresponsive in human time, an advantageously simple and highly robust andflexible hierarchal event processing mechanisms are employed in place ofthe standard pre-emptive or cooperative threaded systems employed in thecurrent state of the art interoperability operating systems and theirassociated runtime environments.

Consider as an example a slide show Dart application. Although anyprogramming language may be used, in one implementation, the slide showDart is written in C++ program code language, using the DartFramework(FIG. 11 102), also written in C++, and compiled and linked using theDartTools (FIG. 12 200). The C++ source code also contains Dart specificC++ pragma statements that extend the applications that can be createdto include all the DartProcedures, Renditions, and otherinteroperability extensions necessary for the Dart applications to findand inspect other devices, send optimized copies or parts of themselvesbased on the said inspection, cause the optimized copies or parts toexecute on selected discovered devices, and then interoperate by way ofevent queues on all involved devices. Such devices may be event drivenwith the events automatically serialized and/or synchronized so that allcomponents of the application are operating in a highly cooperative androbust manner in order to express the intent of the interoperableapplication as encapsulated in a Dart (FIG. 9). In the preferredimplementation, a DartMaster (FIG. 12 230) generated directly from theDartTools (FIG. 12 200) that will consist of a single MasterRendition(FIG. 11 113) derived object, which provides a main method as thestarting point for execution which proceeds to build a list maintainedby the MasterRendition object which contains references to otherRendition derived objects (FIG. 11 114).

DartMasters are typically built by programmers in C++, or other highlevel languages using tools and a framework that target theDartInstructionSet. To cover the full range of devices that a Dart mayfind itself running on, the MasterDart will typically contain contentand procedures used by the MasterPlayer to generate between one and fivedifferent Renditions.

Logically, Renditions can be thought of as separate programs, orexecutable images, where in the preferred implementation, only one ofwhich will be chosen to run at any one time on any one device.

Physically, the Renditions often advantageously share most of theirdata, code and content components so that there is advantageously agreatly minimized amount of duplication in the actual DartFormat binaryimage or file.

A Dart's DartSource (FIG. 3 100) should also include procedures that runon a device to determine which Rendition is best to run on that device.

For example, a slide show Dart containing a collection of pictures mighthave the following three Renditions: (i) a simple text Rendition fordevices that only have a single line or a few lines of text display andthis Rendition scrolls through the name of the slide show and a list ofthe names of the slides included since it cannot possibly show theimages themselves; (ii) a small screen Rendition such as may be suitablefor cell phones and PDA's with small screen dimensions and limited CPUpower; and, (iii) a high-end large screen Rendition that shows largepictures with multiple display options, indexes and transition effects,as may be suitable for running on lager screen personal computers (PCs)or home entertainment systems.

In one particularly advantageous operating mode, when a Dart containingmultiple Renditions first starts running on a device, a Dart procedureuses the Profile Instruction of the DartInstructionSet to inspect thedevice profile built into the DartEngine on that device to determine ifthe device has all the features necessary to run the most advancedRendition. If it does, then the most advanced Rendition will be run. Ifit does not, then the device profile is procedurally checked againsteach less advanced Rendition until one is found that can run effectivelyon the device.

Note that target devices that do not have a DartEngine built-in can beaccessed by or through any device with a DartEngine that is built withintimate knowledge of the target device and the ability to proxy for andvirtualize the operations of the DartEngine for the device over aconnection to the target devices. An example is the use of a printerwithout a DartEngine which is never-the-less reachable by a device thatwants to print on it through a personal computer which is itself runninga DartEngine which exposes its printing resources through thePROFILE_INSTRUCTION. Any other initiating device with a DartEngine willhave access to the printers available to the personal computer so longas the DartEngine running on the PC exposes the printing capability andaccess through the profile and print methods of the hardware abstractionlayer (HAL) object of the DartEngine on the personal computer.

When a Dart is asked to send itself to another device, one of the useroptions in most Darts will be to have the set of Renditions sent limitedto those that can run on the target device. In this way the user canlimit the transmission time to, and the memory requirements on, thetarget device. Of course the new device will not then have higher levelRenditions to resend to more capable devices.

Since a great deal of content, and the application programs that createand edit the content is PC or Internet based, it is important that Dartcontent be able to import and export content in a form native toexistent software systems.

Darts may advantageously contain standard format data for JPEG pictures,any other forms of data or content that are likely to be encountered,and the like, and menu options can be built into the Dart content thatwill import and export pictures, video, audio, text, and XML documentsthat tie various components together.

The DartInstructionSet optionally but advantageously contains JPEG andother standard media format decompression and playback instructions sothat the CPU intensive decompressing tasks are implemented in efficientfunctions compiled as part of the DartEngine into the native instructionset of the DartDevice's CPU. These instructions also limit the amount ofcode that must be included in Darts since the DartEngine does much ofthe decompression and display as part of executing theDartInstructionSet. Similarly there are XML and other text parsinginstructions to limit the amount of application code and provide nativecode speed advantages when parsing text, RTF, XML and other text baseddocuments.

In addition, PC applications can easily be adapted by theirmanufacturers to import and export Dart content directly. Darts cancontain content access APIs and control API's that allow otherapplications and Darts to programmatically enumerate and extract contentParts such as JPEG pictures, and audio clips along with the text namesand descriptions of the Parts.

Dart control APIs, along with text descriptions of the API functions canalso be enumerated and accessed to control the operations of the Dartsthemselves from other Darts or devices allowing remote control of Dartsand the devices they operate.

Embodiment of a Procedure for Porting a Dart Engine to a Device

A procedure for porting a DartEngine to a new device is now described.The example assumes that the Dart Engine is a C++ code Dart Engine, butthat does not distract from the generality of the procedure.

First, create your own Hardware Abstraction Layer (FIG. 4 3020, FIG. 22650, FIG. 27 650) object by inheriting from the halBase object or in anyother way such as by direct creation.

Second, fill in the functionality of the halBase member functions whichinclude such functions as allocating a single consecutive block ofmemory of a given size, returning the time in milliseconds, moving abitmap in one of the three standard internal formats, or compile timevariants to the screen at a given location. The halBase class alsoincludes a virtual function for getting profile information words thatmust be provided to allow Darts running on the DartEngine to determinethe CPU, memory, screen, communications, sound, printing, and othercharacteristics of the actual device. Of particular benefit is to designand build a programmatic interface through the use of the OEM_Functionmethod of the halBase object to expose any unique capabilities, contentor functionality of the device not otherwise exposed by the pure virtualmethods of the base halBase class.

Third, create a device specific DartEngine object (FIG. 22 600) byinheriting from the playerBase object, or create it directly withoutbenefit of inheritance.

Fourth, build the device's DartPlayer executable (FIG. 22 600) whichemploys the DartEngine object, which includes a loop that calls theDartEngine's Process( ) member function (such as for example, aplayerBase::Process( ) member function (FIG. 23 4003, FIG. 25 611)) in aloop until a predetermined condition, such as a non-zero value, isreturned. All synchronous (FIG. 15 8010) and asynchronous operations(FIG. 16 Asynchronous Event Processing show inside the dotted box) ofthe DartPlayer including communications are driven by the executionthread of this simple loop so that a multithreaded OS is not needed.

The Dart solution to device interoperability advantageously changes theadaptation and testing complexity equation from an N-squared (FIG. 1,and FIG. 2) to an N-ordered one. Instead of having to consider how a newdevice will share content and control with every other kind ofdissimilar device and implement individual solutions, with Darts oneonly needs to port the DartEngine into a DartPlayer specific to thedevice (FIG. 10).

One will need to implement functions that understand all thecommunications channels that the device has, and build routines toreport the CPU speed, screen size and the like, but one never has toconsider how a device will share content and control. The Dart contentdoes that instead of the device's built in software.

It will be appreciated that this porting provides a Dart that is amanageable N-order solution rather than an N-squared solution, and thatthe porting of the DartPlayer is done only once for each new device andfurther that it is only necessary to develop a Dart once for each newapplication so that each new device or application requires only oneadditional unit of work.

Embodiment of a DartPlayer

A description of one embodiment of a DartPlayer (FIG. 22 500),implemented for example in C++ language and compiled using conventionalprogramming tools to the native instruction set of the DartDevice'sprocessor or in some embodiments central processing unit (CPU) follows.

The playerBase class from which the DartPlayer class inherits, executesDart programs which are a series of opcodes with parameters. Basically,the DartPlayer may be analogized to a microprocessor, and theDartInstructionSet analogized with a set of machine instructions;however, in the case of Darts most of the instructions are much morehigh level than in the typical microprocessor. The instructions nativeto the DartPlayer may for example, include single instructions thatdecompress JPEG pictures into a buffer, move pictures in buffers to aparticular place on the physical screen in the correct format, and savethe entire state of the running DART and all its code and data. Inaddition a full set of 2D graphics primitives, advanced user inputprocessing, and audio decompression processing instructions mayadvantageously be built into the DartInstructionSet.

When a Dart wants to: (i) send itself, (ii) send an optimized version ofitself, (iii) request a control panel, (iv) request a DART procedures,or (v) request data from another device, it uses anEnumerationInstruction or puts an EnumerationEvent on an EventQueueusing a BuiltinInstruction. The Enumeration instruction or EnumerationEvent causes the player to call a halBase enumeration member functionthat gets the name and description of each Dart device it can make aconnection to. Optionally, a DartProcedure can be sent to each suchdevice, which runs the procedure, and itself returns a procedure thatruns on the originating device. Thus EnumerationInstructions in concertwith the general processing, mathematics, and test and controlinstructions can be used to effectively inspect any connected device tosee if it is capable of carrying out most any desired set of functions,or contains code, data or content of use or interest.

Since the function of the DartProcedures or Darts sent to other devicesand received back can contain code to do most anything, thisfunctionality advantageously is all that is needed for a wide range ofinter-device cooperative tasks. In most instances, it is up to theperson or programmer porting the player to a device to decide what typesof physical connections, protocols, security measures, and the likedesign choices to take when making a connection to other devices. In oneparticularly advantageous implementation, the DartPlayer assumes thatany communication block received is error corrected and has alreadypassed whatever security requirements the porting programmer built intothe halBase virtual functions. Advantageously, aside from the low levelsending and receiving of such blocks in the device specific functions ofthe HAL, the functionality for setting up secure communicationssessions, sending events with associated data, code and content, and theautomatic requesting and processing of blocks that have been lost intransit, the automatic recovery of sessions temporarily lost betweendevices is all performed in the portable code portions of theDartEngine. This greatly reduces the amount of work needed to supportvarious communications protocols on a DartDevice and greatly improvesthe reliability of implementations between devices because the samesource code base is used to port the portable parts of the DartEngine.

Embodiment of the Use of a Dart Instruction Set for Physical Discovery

Embodiments of the DartInstructionSet advantageously primarily dealswith device, service, and resource discovery and inter-devicecommunication at a very high level of functionality. In someembodiments, the DartInstructionSet deals exclusively with device,service, and resource discovery and inter-device communication at a veryhigh level of functionality. While a running Dart can inspect a deviceprofile to determine actual communication characteristics such ascommunication speed and latency, advantageously generally a running Dartdoes not care or need to know whether the actual communication betweendevices is HTTP, TCP IP, USB, 802.11b, or some other protocol, or ashared memory card, a physically transported memory card or any otherphysical medium or protocol.

Similarly, physical discovery of other devices, services, or resourcesor authentication and security can to be built into the halBase overridefunctions by the people specifying and implementing the port.

One advantageous way to build such a port is to create as part of theplayerBase based DartPlayer, a user editable “friendly” device list.This way the user can specify the Internet Protocol (IP) addressesand/or network names and/or passwords needed to access all the devicesshe wishes to allow the device to have access to which are otherwiseundiscoverable, difficult to discover or difficult to pinpoint amongother numerous networked devices, through other means.

Embodiments of the invention may advantageously be made so that allDartPlatform characters are 32-bit words to advantageously accommodateany character representation system such as Unicode or multi-byte,without any need for special handling inside by Dart procedures. It isup to the halBase derived HAL implementing object to translate to andfrom these 32 bit characters to match the native capabilities of theparticular device. However, the invention is not limited to anyparticular bit word size and larger or smaller bit word sizes may beused.

Devices which wish to do device, resource, and/or service discoveryshould comply with standards that are in place. The Dart Enumerationinstruction interface or SignalEvent Builtin/Instruction (FIG. 20 670)processing advantageously hides the differences between the variousdevice discovery standards for Dart applications, while helper functionscan be provided for use in the hardware abstraction layer, to aid inbuilding compatible support for IP, Infrared (IR), Bluetooth, 802.11x,and other connected or connectable devices or systems.

On at least one exemplary DartPlatform, most all adaptation of content,data and procedures are advantageously performed by the Dart instead ofbeing built into the engine. It is the Dart itself that decides whatRendition will run best on the target device. And it is the proceduresin the Dart that adapt the content playback for connected DartDevices.

Advantageously, with the DartPlatform, there is no need to plan forevery application and other device that a new device will need to workwith because the protocol between devices is very simple, yet powerful.One DartDevice can send a DartProcedure over to any other DartDevicewhich automatically run in the target DartDevice and then returns anEvent or procedure that is automatically run or processed on theoriginating device. The DartDevices do not need to know whether theprocedures or Darts they are running are delivering data, anapplication, a control panel or just inspecting capabilities.

In at least one particularly advantageous embodiment, the DartPlatformdoes not use either the Client/Server or Peer-to-Peer modes ofinter-device interoperability. Instead it advantageously uses theRecruitment model.

Recruitment and the recruitment model and method allows Dartapplications to automatically extend their reach or connectivity acrossa multitude of differing connections and devices. Recruitment is aradical departure from Client/Server or Peer-to-Peer connectivity orcommunication and bestows unique and highly desirable properties to theDartPlatform.

Embodiments of the Recruitment model allows Dart applications to formteams of devices around the application in the same manner that peopleform teams to get a job done. If one is a construction contractor whowants to build a house, one locates carpenters and lumber suppliers withthe skills and resources needed for the construction job. Then one worksas a team with the suppliers and carpenters thus recruited to get thelumber to the carpenters and then to coordinate the building of thehouse according to the intent of the contractor who initiated thebuilding of the house. Similarly, a Dart application can reach out andinspect the resources of all DartDevices it can communicate with overany existent communications medium by sending procedures thatautomatically get run on target devices. A running Dart itself finds,qualifies, and forms a team of DartDevices, sending any content and codeas needed to each DartDevice in the team to effect the application forthe user. Device user involvement is not required.

Darts extend their operation across the Recruited DartDevices andeffectively run inside all DartDevices simultaneously. The result is asystem that advantageously does not require any central control deviceas in Client/Server configuration or operation. Advantageously, the Dartsystem does not require programs related to the mission of theoriginating Dart to reside on any but the originating device, as opposedto Peer-to-Peer configuration and operation. And the device user neverhas to think about media formats, communications protocols, nor loadingdrivers. The Recruitment model also allows Dart applications and data tospread themselves from Dart device to Dart device whenever anotherDartDevice is Recruited into a team. Thus advantageously, distributionof Darts and their encapsulated content, programs, and data can occurthrough usage or without any explicit loading or saving of applicationsor their associated data. Security may often be a necessary part of theDartPlatform to prevent the spread of malicious or malfunctioning Dartsor other code and data. This security partially comes in the form ofrequiring a potential recruited device to accept or reject recruitmentby not allowing communications, or simply declining to run procedures orDarts from other devices.

In one particularly advantageous implementation, the Recruitment modeland method is based on the DartInstructionSet. The procedures areexpressed using the DartInstructionSet which is optimized for deviceinteroperability, multimedia user interfaces, and the use of theRecruitment model and method.

Advantageously, most any high level language can be compiled to targetthe DartInstructionSet. In one particularly advantageous embodiment atthis time, the high level language used is C++ with extensions addedthrough the C++ pragma facility. These advantages derive from the widespread current use of the C++ programming language, but the invention isnot limited to any particular language, and may be equally or betterimplemented with other languages either now existent or to be developedin the future, including for example improvements, enhancements, and/orextensions to the C++ programming language.

Additional particular embodiments for Interoperability Security Model ora more specific embodiment, DartSecurity Model are now described. In oneembodiment (1) , the invention provides a method for limiting access toand or the understanding of resources or capabilities of aninteroperability application package and or resources or capabilities ofinteroperability devices, the method comprising: (1) forming a basis forsecurity using at least a plurality of the following steps: (a)automatically collecting an entropy state measure associated with thedevice; (b) generating a key pair; (c) generating a device Id; and (d)storing the key pairs, device id, and the entropy state measure on thedevice for continued use whenever the device is active; (2) formingrules for allowing or preventing the limiting access; and (3) using theformed basis and formed rules to provide a security task at least forthe device.

In another embodiment (2) , the invention provides a method for limitingaccess to and or the understanding of resources or capabilities of aninteroperability application package and or resources or capabilities ofinteroperability devices, the method comprising: (1) forming a basis forsecurity in at least one of the following steps when a device firststarts operation: (a) automatically collecting of entropy; (b)generating a public/private key pair; (c) generation of a unique deviceId; and (d) storing the public and private key pairs, device unique id,and the entropy state for a random number generator on the device forcontinued use whenever the device is active; (2) forming rules forallowing or preventing the limiting access; (3) use of the formed basisfor one or more of the following security tasks: (a) enforcing the rulesfor access to devices, applications and resources; (b) securing therules themselves so that they cannot be modified by an unauthorized useror agent; (c) securing the storage of the public and private key pairs,device unique id, and the entropy state for a random number generator;(d) securing operating parameters or data to be shared betweenapplications and devices; (e) securing communication channels; (fencrypting resources so that they can only be understood or used by aparticular device or set of devices, and/or a particular application orset of applications, and/or when accessed using a shared secret; and (g)generating universally unique ids and using them for identifyingdevices, applications, data formats, collections, records, individualmedia files, or even individual data items valid across all devices andor applications and or datasets or items for all times.

In another embodiment (3) , the invention provides the method of (2) ,further comprising: revoking or modifying the rules used to limitaccess.

In another embodiment (4) , the invention provides the method of (2) ,wherein the limiting access to and or the understanding of resources ofcomprise one or more of the following in any combination: (1) enforcingrules for which devices are allowed to communicate with each other andor for what purposes devices are allowed to communicate; (2) encryptingdata that flows between devices so that other devices which might haveaccess to the data will not be able to understand or make use of thedata; (3) signing packages of data, content, code or other resources toensure that the packages have not been modified since the signing tookplace; and (4) managing digital rights to enforce a set of rules for thehandling of data, code content, or any other digitally representableresources with regard to one or more of the following actions: sharing,copying, printing, displaying, performing, distributing, playback,rendering, executing, modifying, or any combination of these.

In another embodiment (5) , the invention provides the method of (2) ,wherein one or more of the following are true: (a) the applicationpackage conforms to an interoperability format or the Dart Format and oris a Dart; and (b) the limiting access is performed at least in part byan Interoperability Engine or is performed at least in part by aDartEngine or by a DartPlayer.

In another embodiment (6) , the invention provides the method of (2) ,wherein the forming the basis for security is performed only once forany particular device.

In another embodiment (7) , the invention provides the method of (2) ,wherein the forming the basis for security is comprised of the followingsteps when a device first starts operation: (a) the automatic collectionof entropy state; (b) the generation of public/private key pairs; (c)the generation of a unique device Id; AND (d) the storage of the publicand private key pairs, device unique id, and the entropy state for arandom number generator on the device for continued use whenever thedevice is active.

In another embodiment (8) , the invention provides the method of (2) ,wherein the forming rules for limiting access further comprise: (i)collecting the unique ids of devices and/or applications which wish tointeroperate (ii) collecting the set of access rights needed orrequested for the interoperation; (iii) obtaining permission to allow orthe directive to disallow the range of access rights; (iv) creating aunique id for the set of access rights; and (v) storing the unique idand set of access rights on all devices and or in all the applicationpackages which are to interoperate.

In another embodiment (9) , the invention provides the method of (7) ,wherein the basis of is used for the generation and storage of uniqueids.

In another embodiment (10) , the invention provides the method of (2) ,wherein use of the basis is used for one or more of the followingsecurity procedures alone or in any combination: (a) enforcing rules foraccess to devices and resources; (b) securing the rules themselves; (c)securing communication channels; (d) encrypting resources so that theycan only be understood by a particular device or set of devices, and ora particular application or set of applications, and or when accessedusing a shared secret; (e) generating universally unique ids and usingthem for identifying devices, applications, data formats, collections,records, individual media files, or even individual data items validacross all devices and or applications and or datasets or items for alltimes; and (f) revoking or modifying the rules used to limit access.

In another embodiment (11) , the invention provides the method of (2) ,wherein the resources or capabilities are one or more of the resourcesor capabilities in the set consisting of: (a) a data or data sets of anyform; (b) a code of any form; (c) a content of any form; (d) acapability to communicate in any form and to any other device; (e) acapability to control any aspect of the device or any other device; (f)a capability to control any aspect of the device's operation and or ofany other devices' operations; (g) a capability to render and/or printand/or display and/or modify and/or copy and/or distribute data or anyother element which can be represented by a binary sequence or set ofbinary sequences; (h) a capability to collect information in any digitalform by any means; (i) a capability to physically change the position,disposition, properties and or location of physical objects; (j) acapability to run code and process data; and (k) any combination of theabove.

In another embodiment (12) , the invention provides the method of (2) ,wherein the method applies to the applications whether they are storedas an application image and or running on one or more devices.

In another embodiment (13) , the invention provides the method of (2) ,wherein the basis is implemented in a manner where entropy gathering andthe random number generation is seeded and then used to generate aunique id is automatically initiated when the device is first turned onor is made use of so that the manufacturer does not need to assign orinstall different software or hardware to assure that each device has auniversally unique id.

In another embodiment (14) , the invention provides the method of (2) ,wherein the entropy is gathered by collecting via the execution ofsoftware running on a processor timing or data or counter or hardwarestate values which are at least partially generated by any one or moreof: quantum electrical effects, communications or other electricalcircuits running on a clock unsynchronized to the processor clock, aircurrents, or any other physical phenomena.

In another embodiment (15) , the invention provides the method of (7) ,wherein the basis is used for the generation and storage of unique ids.

In another embodiment (16) , the invention provides the method of (15) ,wherein the if a device loses its unique id or private or public key andcannot be recovered, it will be necessary for a new unique id andprivate and public key to be generated for the device before the devicecan again be used.

In another embodiment (17) , the invention provides the method of (16) ,wherein after a new unique id and private and public key is generatedfor the device, the device is considered for all purposes of security tobe a new device.

In another embodiment (18) , the invention provides the method of (2) ,wherein a device or application unique id is revoked, the unique id isplaced on a list of revoked unique ids that is maintained anddistributed on teamed devices and or other devices so that the device orapplication will no longer function as part of a team or teams when ittries to interoperate with any of the devices with the revoked unique idwhich is on such a list.

In another embodiment (19) , the invention provides the method of (18) ,wherein when a revoked device forms a communication link with anotherdevice which has the revoked device's unique id on its revocation list,the software driving the communication link will initiate thedestruction of the unique id and public and private keys of the revokeddevice.

In another embodiment (20) , the invention provides the method of (18) ,wherein the revoked id list is maintained and distributed using thesocial synchronization procedure.

In another embodiment (21) , the invention provides the method of (18) ,wherein only devices with certain permissions, unique id, or sharedsecret is allowed to place a device or application id on any revocationlist.

In another embodiment (22) , the invention provides the method of (19) ,wherein one or more master devices with certain permission, unique id orshared secret or has the capability to communicate of a special securechannel can initiate the destruction of the unique id and public andprivate keys on the revoked device over the special secured channel.

In another embodiment (23) , the invention provides the method of (22) ,wherein the master device is at the headend of a mobile phone networkcan initiate the destruction of the unique id and public and privatekeys on the revoked device over the mobile data network or a sidechannel thereof.

In another embodiment (24) , the invention provides the method of (2) ,wherein the rules are stored as lists of the unique ids of devices andapplications associated with a certain level of security.

In another embodiment (25) , the invention provides the method of (24) ,wherein the rules are stored as lists of the unique ids of devices andapplications associated with a certain level of security, and each levelof security is associated with certain access rights binary flags, witheach binary flag associated with permission to access a particularcategory of data, resource, code or capability.

In another embodiment (26) , the invention provides the method of (25) ,wherein a device or the runtime context in which an executable image isrunning maintains the access rights flags values which determine whichtypes of access or elements are to be allowed for the executable imagesrunning on that device or the executable image running a particularruntime context.

In another embodiment (27) , the invention provides the method of (26) ,wherein an interoperability engine running an interoperabilityinstruction set checks any or all of the flag values whenever anapplication attempts to access the elements or capabilities whichcorrespond to the flags, and does not allow such access if the flagvalues indicate that such access is not to be allowed.

In another embodiment (28) , the invention provides the method of (27) ,wherein one or more of the following are true: (1) the interoperabilityengine is as described elsewhere in the detailed description or is theDartEngine; and (2) the interoperability instruction set is as describedelsewhere in the detailed description or is the DartInstructionSet.

In another embodiment (29) , the invention provides the method of (25) ,wherein the rules embodied by the stored lists are generated using oneor more of the following method: (1) default mappings for flags tosecurity level numbers are supplied by the manufacturer and optionallymodifiable by possibly secured interfaces on the manufactured device;(2) the security level and or flag values that are needed for aparticular executable image to carry out one or more operations of aninteroperability application software package is embedded in the image,which is optionally signed to ensure that the security level and or flagvalues have not been tampered with; (3) when an executable image is tobe loaded by an interoperability engine, the embedded level and or flagvalues are compared to the stored rules to see if permissions to accessall the needed access is allowed; (4) if all permissions are allowed theexecutable image is loaded to run in a context that contains the accessflag values embedded in the executable image; (5) if during execution,assess to any resource or capability for which the flags in the contextdo not allow access causes the interoperability engine to immediatelystop execution without allowing the access; and (6) if any of thepermissions are not allowed the interoperability engine will not load orrun the executable image.

In another embodiment (30) , the invention provides the method of (2) ,wherein the method is used for one or more of the following purposes oroperations: (1) assurance that the content, code, and or data hasn'tchanged since some predetermined date or event; (2) assurance that thecontent, code, and or data has been inspected or otherwise certified bya particular party as one or more of the following in any combination:(a) robust; (b) devoid of viruses, malicious code, of other undesirablefeatures or operations; (c) devoid of a predetermined or dynamicallydetermined particular kind of objectionable materials; (d) to have beencreated by or received from a particular third party; and (e) being ornot being any one or combination of the above; (3) protection from thespread of software viruses or malicious computer code; (4) protectingintellectual property rights including but not limited to copyrights andtrademarks; (5) enforcing intellectual property rights and agreements;(6) obfuscation of the rules, unique ids, code or data on a device sothat it is difficult for someone to find and or understand the meaningof the obfuscated items; and (7) the generation of universally uniqueids to assign to data sets, data items, categories of items or any otherelement or grouping of elements or ideas which needs to be uniquelyidentified.

In another embodiment (31) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for limiting access to and or the understanding of resources orcapabilities of an interoperability application package and or resourcesor capabilities of interoperability devices, the program moduleincluding instructions for: (1) forming a basis for security using atleast a plurality of the following steps: (a) automatically collectingan entropy state measure associated with the device; (b) generating akey pair; (c) generating a device Id; and (d) storing the key pairs,device id, and the entropy state measure on the device for continued usewhenever the device is active; (2) forming rules for allowing orpreventing the limiting access; and (3) using the formed basis andformed rules to provide a security task at least for the device.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XVIII. Social Synchronization Interoperability Method/Dart SocialSynchronization

Synchronization of data sets or operations across a plurality of devicesis most often done with traditional methods where there is assumed to bea single master device to which all other devices directly synchronizetheir data sets. These methods provide a highly robust synchronizationsystem in the corporate and government networks of general purposecomputers that are always connected to each other with reliable highspeed networks administered and maintained by trained professionals.Most current synchronization of individuals' devices such as mobilephones, PDAs, and digital cameras is now conventionally performed usingthese same direct to single master device methodologies where the masterdevice is a personal computer or an internet connected server; however,having to require synchronization to single master is very limiting forthe world of intermittently connected, often mobile devices for manyreasons including,

-   -   1. All devices must share at least one communication protocol        with the master device.

2. Some mobile devices can only synchronize when they are in closeproximity to the master device because they only have limited rangewired or wireless direct connections suitable for synching data setswith the master.

3. The master device provides a single source of failure for alldevices.

4. The individual users without training must concern themselves withloading device specific synchronization software on the master deviceand configuring and maintaining this software.

5. There is no easy way for mobile devices to synchronize directly toeach other, even when they are in dose proximity of each other and sharea common communications protocol.

The inventive SocialSynchronization is a particular subset of themethods that can be used to synchronize DartDevices that provides manybenefits over the conventional mastered synchronization methods commonlyused in the current state of the art. Recall that SocialSynchronizationis an efficient and easy to administrate method for synchronizingspecific sets of data and or operations across any number of devices andprotocols without the need for every device to contact a master device,or for any device to act as a master. SocialSynchronization of devicesand content is similar to the way humans share information and tasks andis an advantageous alternative to mastered synchronization techniquesmost often used in the current state of the art.

Some of the benefits and advantages of the inventive system and methodover the conventional mastered synchronization methods commonly used inthe current state of the art include:

-   -   1. Synchronization can be performed for teams of devices where        there is no need for all the devices to be able to communicate        directly to a single master; rather, synchronization can be        carried out so long as there is any path of communication        through any number of devices' between each other, and where        such paths do not need to be established simultaneously.    -   2. Mobile devices do not need to be able to direct connect to a        master; rather, they just need to be able to direct connect to        any one of the mobile devices in a team of devices established        for the synchronization.    -   3. There is no master device and so no single source of failure.        Teamed devices which fail can be replaced by new devices which        can then easily synchronize to any other devices in the team.    -   4. There is no need to install device specific software on any        of the devices or to actively maintain or configure such        software; rather any device in a synchronization team can        directly add any other device to the team.    -   5. Devices can synchronize directly with any other devices in        the team to which there is any path of connectivity through any        sequence of connects between devices, even where such        connections are not available simultaneously.

SocialSynchronization works somewhat like teams of humans who shareinformation with other humans who they encounter, who in turn often thenshare that same information with still other humans. A human analogy canalso be used to illustrate the limitations of mastered synchronization.Imagine a company with a large number of employees who have no way ofsharing information about all the activities and initiatives of theircoworkers except by talking to a single boss. There are just too manyinteractions necessary to distribute all the needed information for itto be practical for all knowledge that needs to be shared to go througha single boss, or even through a fixed hierarchy of bosses. Individualemployees need to share information of common interest directly to eachother as they encounter each other and expect that information will getdistributed to others with a need to know through intermediaries. Ineffect, these methods of social synchronization, while not perfect, arevery effective methods of distributing information among a group or teamof people or devices where there are a number of ongoing intermittentand possibly irregular connections taking place between humans orbetween devices.

FIG. 30 shows an example of how SocialSynchronization is performed inone preferred embodiment. The example shows the synchronization ofsimple changes to a simple contact list, but it should be appreciatedthat the synchronization can be performed for complex databases and orthe synchronization of not just data but also of the operations of andresults of operations of a team of DartDevices and or Darts.

In the example of FIG. 30, there is an initiating device, A, with aContactList Dart application which initially contains two entered names.The two other devices, B and C initially have no knowledge about theexistence or characteristics of the ContactList Dart running on A. Inone preferred embodiment, the ContactList Dart running on Device Aenumerates all the devices with which it can share the ContactList Dartthat contains and controls the list of contacts. The ContactList Dart,uses the method of Dart Recruitment and optionally the method ofrenditioning. The Recruitment by the ContactList Dart on Device A ofDevice B is initiated by the user by selecting a “Team to Other Devices”menu item. The user is then presented with a list of devices currentlyreachable and suitable for the teaming of the ContactList as determinedby one or more DartProcedures or Darts sent to run on the reachabledevices, and return information about their suitability for becomingrecruited. The user selects device B from the list which results in theContactList Dart saving a Dart containing one or more or all of itsrenditions and the data that constitutes the content list elements. Thissaved Dart is sent to B as part of a RunDart type event. When the Dartruns on device B it saves itself so that it will be available on B evenafter the device is disconnected from device A or powered off. The Dartnow running across both devices makes an entry, if none yet exists, ineach device's SocialSync uid list. The entry contains the uid of theContactList with the two names to be shared that was generated by theDart when it first created this particular list of contact names. Uidsare generated using the DartSecurity infrastructure described elsewherein this document. The entry also contains the uid of the ContactListDart. The DartTools automatically place a uid into every Dart instancegenerated.

Similarly at some time after device A finishes its permanent teamingoperation with B, device B is used to permanently team to SocialSyncObjectUid/DartHandlerUid lists with an entry for the 128 BitUidA thatidentifies the particular teamed logical instance of the contact list.The entry also contains the uid of the Dart stored on the same devicewhich can be used to carry out the synchronization of the contacts forthat list. Note that in general, the handler Dart can perform anysynchronization decisions or operations or rules expressible as a Dartand is not limited to simple data synchronization operations as in thisexample.

The three devices now all contain the SocialSync entries containing theuids and two names of the original contact list even though device C hasnever communicated directly with Device A.

In the example, after the devices are teamed the connections are closedand a new name is added to the list on Device C by the user who runsContactHandlerDart Z. The three block diagrams show the state of thethree devices immediately after the third name is added on Device C.

Next the ContactHandlerDart Z or any other Dart running on C initiates aconnection to Device A. While setting up the connection, the DartEngineon C automatically compares the SocialSync uids on the two devices anddetermines that they share the 128 BitUidA uid. The DartEngine on C willautomatically start up ContactHandlerDart Z and it will coordinate thesynchronization of the data whether stored in Darts or storedseparately. Now devices A and C contain all three names in their contactlists.

After Device B connects to either A or C it will similarly runContactHandlerDart Y which will carry out the synchronization of namesbetween device B and the device it connects to. Note that event though Ahad never communicated with C before syncing, they knew that they neededto sync and how to sync because of the Dart and data passed by theintermediating device B during the individual Teamings.

It logically follows that any number of devices can be easily added tothe team transitively by any device already teamed even if the newdevices knows nothing about the Contactlist/ContactHandler Dart ororiginating Dart until Teamed. So long as the ad-hoc connections betweenthe teams of devices are sufficient, all the devices automaticallymaintain a high degree of synchronization of the shared list of contactseven in the absence of any master device.

The inventive system, method, and computer programs and computer programproducts also provide a platform having Serialization andSynchronization. The intra-device DartRuntime shown in FIG. 15, FIG. 16and FIG. 18 along with the inter-device DartRuntime shown in FIG. 17together create a single event driven DartRuntime where all operationsare advantageously carefully serialized and synchronized across allprocessing units of all teamed devices. FIG. 9 Illustrates how thisserialization and synchronization infrastructure embodied throughout theDartPlatform including in Dart Recruitment, Dart Renditioning, theDartSource, the DartTools, the DartFramework the DartRuntime and theDartEngine. Together these systems and methods and means provide aninfrastructure for ensuring the robust interoperability for Dartinteroperability applications with little effort on the part of the Dartprogrammer needed to set up or administrate the complex interactions andordering of operations across processing units cooperating on andbetween multiple devices, even in the face of communications and othererrors.

Writing Dart applications using the DartPlatform and particularly theDartFramework automatically gives applications binary portability acrossall DartDevices, robust power management, robust application level errorrecovery, the simple and efficient teaming of devices, the intelligentdiscovery and use of resources across devices, and many of the otherbenefits of the DartPlatform, all with relatively little effort on thepart of the Dart programmer. A key element for delivering these benefitsis embodied in the DartRuntime through the largely automaticserialization and synchronization of events as shown in the example ofFIG. 9.

The DartRuntime coordinates the passing or exchanging of Dart Events andassociated files between interoperating and communicating initiating andrecruited devices. Dart Events are automatically copied and/orsynchronized across event queues of any number of teamed devices. Eventsdesignated as synchronized events are robustly distributed across allthe event queues of all devices which share a list of synchronized eventtypes. This is carried out in the example shown in FIG. 9 as follows:

-   -   1. The initiating application, rendition 701 a of Dart 700        instantiates a connection manager object instance on the        originating device 400-1 for a specific unspecified inter-device        cooperative function. The Connection Manager is an instance of a        ConnectionManager class that is provided in the DartPlatform        (FIG. 3) as part of the DartFramework (FIG. 3 102).    -   2. The initiating rendition, 701 a, recruits a team of devices        using Recruitment and Renditioning to create a team of devices        with a shared duplicative list of event types to be serialized        and synchronized over all cooperating devices. This list is        maintained by instances of the Connection Manager 420 on each of        the teamed devices (400-1, 400-2, 400-3, 400-N).    -   3. Whenever any events are to be placed into a queue on any of        the teamed devices that is on the shared list, then the event        will be placed in all the queues (660) of all directly connected        teamed devices first and then delay the placement of the event        in the initiating device's queue until acknowledgement is        received that the event has successfully been placed onto all        directly teamed devices' queues. Since all the other devices        will do the same, the original event is not placed into the        initiating device's queue until all other teamed devices have        the event in their queues, even those devices which are not        directly connected to the initiating device. This ensures that        events are not processed by any of the teamed devices if there        are any errors which prevent the placement of the event on any        of the queues of the teamed devices. Errors are reported by the        serially propagated return of the original event to the senders        with an error flag indicating that the event is returning        status, and with an error code in the returnedValue word of the        Event structure instance.

4. In order to prevent events generated by one device from arriving outof order on any directly connected devices, all events to be placed ondirectly connected teamed devices' event queues are serialized by notallowing any new events to be placed on a teamed device's queue untilacknowledgement is received that the previously sent event has beensuccessfully placed on the queue. An example of where this isnecessarily is where the first event is an ADD_SLIDE type event used toadd a new slide to the shared slide show, and a second event is a showslide (SHOW_SLIDE) type event indicating that the new slide is to beshown. Due to its much smaller size, it is quite likely that the secondevent would arrive on the device before the first and the applicationmight try to show the slide before it was finished arriving. This is whyit is necessary to serialize the events between all directly connecteddevices.

Serialization of events sent to all directly connected devices ensuresthat all events sent from an individual device will be processed in theexact order sent on all teamed devices; however, it is also necessary tosynchronize the exact order of events when two or more different devicesindependently signal events marked for synchronization by virtue ofbeing on the shared synchronized event lists 430 present on all teameddevices.

One preferred method for ensuring that all synchronized events areprocessed in the same order on all devices even when synchronized eventsneed to be signaled independently by two or more devices is through theuse of a MasterSendEvent event type reserved for use by the connectionmanagers. Each connection manager considers the device that recruited itthrough a direct connection as its logical master. In FIG. 9 device400-N's logical master is device 400-3, device 400-3's logical master isdevice 400-2, device 400-2's logical master is device 400-1. Device400-1 has no logical master because it performed the initiatingrecruitment of the team. This makes device 400-1 the team master. Anydevice with a logical master's connection manager will not place eventsto be placed onto its queue into its queue unless it is being placedthere by its logical master, rather the connection manager willencapsulate all the information needed to signal the event into aMasterSendEvent event type event and place that in all its directconnected teamed devices. All such MasterSendEvent events willeventually propagate to the team Master. When the event is to be placedon its queue, the connection manager, knowing that it is the team masterwill attempt to place the reconstituted original contained event ontoits queue. If the contained event type is on the synchronization list ofthe connection managers it will now be propagated to all the teameddevices in a serialized manner just as if it had been generated andsignaled by the team master.

In effect the team master is the only device allowed to sendsynchronized events to be placed onto the queues of other teameddevices. All other devices need to request that the master send anysynchronized events for them. Since all synchronized events are sent bythe team master over a serialized channel, all devices in the team willreceive all events on their queue in the exact same order, preservingthe integrity of the synchronized operations across all teamed devicesas desired.

Note that in one preferred embodiment, that the serialization ofsynchronized events can itself be used to ensure that the connectionmanagers on all devices maintain the exact same list of event types tosynchronize. Also the serialization system can be used to sendserialized and synchronized events of a type that indicates that a newdevice is to be considered the team master. Since the event declaring anew master is serialized and synchronized all devices will receive suchevents in the exact same order ensuring that even multiple devicessending events declaring different new masters will eventually resolveitself with all devices getting the same last event declaring a newmaster so that the last such event processed will win out.

Selected particular embodiments of the invention involving a SocialSynchronization Interoperability Method or the more particular DartSocialSynchronization, are now set forth. In one embodiment (1) , theinvention provides a method for maintaining synchronization of resourcesacross one or more dynamically created teams of homogeneous and/orheterogeneous devices which can be intermittently directly connectedand/or can be indirectly connected through a sequence of directconnections which themselves can be independently intermittently madebetween other teamed devices, the method comprising: (1) running on aninteroperability device an initiating interoperability applicationprogram package of one or more independently executable images whichlogically or physically encapsulates the resource or resources to besynchronized and/or includes all the capabilities needed to collect andmanage the resources to be synchronized; and (2) teaming of otherdevices by the initiating interoperability application program wherein apossibly intermittent connection is made to other devices and theinitiating application spreads interoperability information to the newlyteamed devices.

In another embodiment (2) , the invention provides the method in (1) ,wherein the method further optionally includes: (3) the repetition ofthe teaming of other devices where the interoperability applicationpackages running on any one or more of the teamed devices acts as theinitiating interoperability application program package so that the teamof devices can continue to grow in a limited or possibly unlimitedmanner.

In another embodiment (3) , the invention provides the method in (1) ,wherein the method further optionally includes: (4) synchronizingresources among and between the devices.

In another embodiment (4) , the invention provides the method in (1) ,wherein the method further optionally includes: (3) the repetition ofthe teaming of other devices where the interoperability applicationpackages running on any one or more of the teamed devices acts as theinitiating interoperability application program package so that the teamof devices can continue to grow in a limited or possibly unlimitedmanner; and (4) synchronizing resources among and between the devices.

In another embodiment (5) , the invention provides the method in (3) ,wherein the synchronization method further optionally includes: (a)initiating a communication session between any two teamed devices; (b)comparing the synchronization unique ids on one device with thesynchronization unique ids on the other device; (c) running theinteroperability application program associated with every matchingunique id; (d) spreading the execution of the interoperabilityapplication program across both devices; and (e) performing, by theinteroperability application program which has its operations spreadacross the two devices, whatever synchronization methods are needed tocarry out the interoperability synchronization purpose.

In another embodiment (6) , the invention provides the method in (4) ,wherein the synchronization method further optionally includes: (a)initiating a communication session between any two teamed devices; (b)comparing the synchronization unique ids on one device with thesynchronization unique ids on the other device; (c) running theinteroperability application program associated with every matchingunique id; (d) spreading the execution of the interoperabilityapplication program across both devices; and (e) performing by theinteroperability application program which has its operations spreadacross the two devices whatever synchronization methods are needed tocarry out the interoperability synchronization purpose.

In another embodiment (7) , the invention provides the method in (1) ,wherein the interoperability information includes: (i) one or moreinteroperability universally unique ids to be used to signify thatdevices are teamed for a particular synchronization purpose; and (ii)one or more interoperability application packages associated with theuniversally unique ids which can be executed to carry out thesynchronization purpose.

In another embodiment (8) , the invention provides the method in (6) ,wherein the interoperability information includes: (i) one or moreinteroperability universally unique ids to be used to signify thatdevices are teamed for a particular synchronization purpose; and (ii)one or more interoperability application packages associated with theuniversally unique ids which can be executed to carry out thesynchronization purpose.

In another embodiment (9) , the invention provides the method of (1) ,wherein one or more of the following are true in any combination: (1)the teams are formed at least in part using a device recruitmentprocedure or Dart Recruitment; (2) one or more of the interoperabilitydevices is a DartDevice; (3) the interoperability application programpackage conforms to the Interoperability Format or conforms to theDartFormat and or is a Dart; (4) the independently executable images arerenditions or are Dart Renditions; (5) the resources are one or more ofthe types described as resources by the description of InteroperabilitySource; (6) the interoperability universally unique ids are createdusing the social security procedural basis; and (7) the one or moreinteroperability application program packages are generated using one orboth of the recruitment method or the creationism method.

In another embodiment (10) , the invention provides the method of (1) ,wherein synchronization comprises one or more of the following in anycombination: (1) maintaining independent instances and/or semantics ofcommonly identified database instances and/or anything that can be heldby a database or element contained in a database in such a manner thatchanges made independently to the separate instances are resolved inwhatever ways are needed to maintain one or more of: (a) the integrityof the database as encapsulating the intended data and purposeassociated with the common identification; (b) the elements of thedatabase; (c) the semantics of the database and/or interrelationships ofelements of the database; and (d) the common identity of the instancesof the databases; (2) tracking the adding, deleting or modification ofelements on a lists of items independently stored and or modified onindependent devices; (3) the collecting and or processing and orcollating of information by a team of devices; (4) maintaining lists ofunique ids, shared secrets and security settings needed to control andlimit access to resources or capabilities of devices; (5) inter-devicemanagement of settings effecting the operation of devices or the set ofdevices; (6) configuration management across multiple devices; (7)installation of software or content across multiple devices; (8) takinginventory of devices and or the resources of the devices or reachable bythe devices or for which information is stored and or maintained by oneor more of the devices; (9) software or content distribution or updatingacross multiple devices; and (10) any combinations of the above.

In another embodiment (11) , the invention provides the method of (10) ,wherein the synchronization is done in an intelligent manner at least inpart by the execution of software procedures so that synchronization isdone in a way specific to each device and/or subsets of devices and/orthe environment thereof.

In another embodiment (12) , the invention provides the method of (1) ,wherein the grow in a limited or possibly unlimited manner is optionallycarried out using one or more of the following: (1) limiting by a setfanout value so that each device or application is only allowed to bringa predetermined limited number of other devices or applications into theteam equal to or within some predetermined magnitude relationship to thefanout value; (2) limiting by a set generation limit procedurecomprising the following: (a) incrementing a generation tracking valuecarried by each application package that sends application packages toteamed devices so that each application package sent to newly teameddevices has a tracking value one greater than the sending applicationpackage; and (b) limiting by a set generation stop value so thatapplication packages with a generation tracking value equal to the setgeneration stop value are not allowed to bring new devices orapplications into the team; (3) limiting according the capabilities orresources of the devices; (4) limiting according to the rules asembodied in the code, data and content of the independently executingimages; (5) limiting according to the level of security of potentiallyteamed devices or applications and/or any other required credentials;and (6) any combinations of the above.

In another embodiment (13) , the invention provides the method of (1) ,wherein the purposes are one or more of the purposes selected from thelist of purposes consisting of: (1) maintaining independent instancesand/or semantics of commonly identified database instances and/oranything that can be held by a database or element contained in adatabase in such a manner that changes made independently to theseparate instances are resolved in whatever ways are needed to maintainone or more of: (a) the integrity of the database as encapsulating theintended data and purpose associated with the common identification; (b)the elements of the database; (c) the semantics of the database and/orinterrelationships of elements of the database; and (d) the commonidentity of the instances of the databases; (2) tracking the adding,deleting, or modifying elements on a lists of items independently storedand or modified on independent devices; (3) collecting and or processingand or collating of information by a team of devices; (4) maintaininglists of unique ids, shared secrets and security settings needed tocontrol and limit access to resources or capabilities of devices; (5)inter-device managing of settings effecting the operation of devices orthe set of devices; (6) configuration managing across multiple devices;(7) installation of software or content across any one or multipledevices; (8) inventorying of devices and/or the resources of the devicesand/or resources reachable by the devices and/or for which informationis stored and or maintained by one or more of the devices; (9)distributing software or content and/or updating software or contentacross multiple devices; and (10) any combination of the above.

In another embodiment (14) , the invention provides the method of (1) ,wherein the method is used as an alternative to other synchronizationmethods because less human administration is necessary and/or thesophistication of the humans who do the administration does not need tobe as high as conventional approaches.

In another embodiment (15) , the invention provides the method of (1) ,wherein one or more of the teamed devices are reachable on the Internetas servers running an interoperability engine.

In another embodiment (16) , the invention provides the method of (1) ,wherein one or more of the teamed devices are reachable on a network asservers running an interoperability engine.

In another embodiment (17) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for maintaining synchronization of resources across one or moredynamically created teams of homogeneous and/or heterogeneous deviceswhich can be intermittently directly connected and/or can be indirectlyconnected through a sequence of direct connections which themselves canbe independently intermittently made between other teamed devices, theprogram module including instructions for: (1) running on aninteroperability device, an initiating interoperability applicationprogram package of one or more independently executable images whichlogically or physically encapsulates the resource or resources to besynchronized and/or includes all the capabilities needed to collect andmanage the resources to be synchronized; and (2) teaming of otherdevices by the initiating interoperability application program wherein apossibly intermittent connection is made to other devices and theinitiating application spreads interoperability information to the newlyteamed devices.

In another embodiment (18) , the invention provides a computer programproduct as in (17) , further optionally including instructions for: (3)the repetition of the teaming of other devices where theinteroperability application packages running on any one or more of theteamed devices acts as the initiating interoperability applicationprogram package so that the team of devices can continue to grow in alimited or possibly unlimited manner.

In another embodiment (19) , the invention provides a computer programproduct as in (18) , further optionally including instructions for: (4)synchronizing resources among and between the devices.

In another embodiment (20) , the invention provides an interoperabilitydevice maintaining synchronization of resources across one or moredynamically created teams of homogeneous and/or heterogeneous deviceswhich can be intermittently directly connected and/or can be indirectlyconnected through a sequence of direct connections which themselves canbe independently intermittently made between other teamed devices, thedevice comprising: an initiating interoperability application programpackage of one or more independently executable images which logicallyor physically encapsulates the resource or resources to be synchronizedand/or includes all the capabilities needed to collect and manage theresources to be synchronized; means for running, on the interoperabilitydevice, the initiating interoperability application program package; andmeans for teaming of other interoperable devices by the initiatinginteroperability application program wherein a possibly intermittentconnection is made to other devices and the initiating applicationspreads interoperability information to the newly teamed devices.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XIX. Social Security Interoperability Model/Dart SocialSecurity

Recall that SocialSecurity is a particularly simple to administratemethod for forming webs of security between teams of possibleintermittently connected devices. SocialSecurity works in a similar wayto how humans often come to trust one another. The foundation forSocialSecurity is the use of SocialSynchronization to spread unique idsgenerated using the DartSecurity system along with the allowed accessrights which travel transitively from device to device. Devices whichhave never directly communicated will often find that they are part of ateam of devices which are allowed to interoperate with certain accessrights without any need for further gathering permissions.

One of the biggest problems in the current state of the art in securityis that it is so difficult for end users to understand and administersecurity methods that they end up not using them at all. There is adirect relationship between the strength of security methods used andthe complexity of administration. In traditional corporate andgovernment computer networks a high degree of strength was deemednecessary and was necessarily administered by highly trained full-timeprofessionals. The fact that such networks were rarely reconfigured madethe amount of administration reasonable for the professionals. Thesesame conventional security methods are often applied to the evolving andgrowing world of mobile devices used by individuals with no training andno time or patience for full time administration of securing networks ofdevices that would need constant administration as mobile devices comeand go. The inventive SocialSecurity method, while perhaps not as strongas the traditional methods employed in corporate and governmentnetworks, does deliver a good level of security, while being so easy toadministrate for devices, that most people will actually use it wherethey often would not use or turn off the traditional methods. ThusSocialSecurity advantageously delivers a good level of security insituations where no security would otherwise be used.

Email is a good example. Traditional secure email protocols have beenavailable and built into the majority of existing email users' clientsoftware since they began using email; yet, few email end users make useof these secure email protocols. One reason is that the administrationneeded to get, secure and use certificates and ensure that everyone theywant to email to securely has also performed the administrationnecessary to get, secure and use certificates is too complex andtedious. Another example is that it is so difficult to set up securityon wireless access ports that many home wireless networks are leftunsecured.

SocialSecurity is similar to human models for securing interactions. Onelearns to trust a new employee of a company with company proprietaryinformation because the new employee was introduced by others in theorganization. It is likely that old employees will began trusting thenew employee without ever talking directly to anyone with originalknowledge that the new employee is a bona fide employee of the company.This web of trust is established without any need for centralcoordination or tracking. Devices such as cell phones, printers, PDAs,laptops, digital cameras and portable music players often have media,information or control to share, but they are never all connectedtogether at once or to any one same central source. Yet it is desirableto use security methods to protect the integrity of the information ondevices from improper use or corruption by other devices or software.The SocialSecurity method is a particular subset of the inventiveInteroperability Security or inventive DartSecurity methodologies of theDartPlatform where access rights are passed from device to deviceautomatically establishing webs of trusted devices and softwareapplications or Darts with a specific set of access rights with aminimum of administration or training required for the users of thedevices.

The SocialSecurity methodology builds upon the DartSecurity methodologydescribed elsewhere in this document. DartSecurity (FIG. 29) ensuresthat every DartDevice and Dart application has a universally uniqueidentifier, uid. Each DartDevice also maintains lists of uids of otherDartDevices which are allowed access rights as indicated by sets ofbinary flags. Each set of binary access rights flags is assigned to alevel.

SocialSecurity is a transitive method for forming and maintaining teamsof devices, and distributing and maintaining the lists of devices withcommon access rights across all teamed devices. FIG. 31 shows aDartDevice on the left side with the DeviceSecurity Lists, a RevokedList, and Device Uid elements of SocialSecurity. The SocialSync Listshown is part of the SocialSynchronization methodology describedelsewhere in this document which is used in a preferred implementationat least in part to spread and synchronize the DartLists transitivelyacross devices.

The right side of FIG. 31 shows an example of the state of old and newDartDevices before and after teaming to help in illustrating theSocialSecurity methodology. In this example before teaming, the OldDartDevice uidA has lists of uids of applications and devices which willbe allowed to interoperate within the access rights of the flagscorresponding to the level of the list. Before teaming the NewDartDevice uidZ's only DeviceSecurity List allows the device itself tointeroperate with itself at Level 9. Ordinarily some of the Dartapplications that are shipped with the Device from the manufacturer needa high level of access to configure the device and or the securityaspects of the device, so the device will grant the applications on theDevice itself a high level of access as indicated by having the device'sown uid in the Level 9 list.

When a Dart on the new device initiates the device's first contact withthe old device and attempts to send and run a Dart generate fromrenditions of itself to run on the old device, the access rights in theDart and of the new device are checked. Of course the old device knowsnothing about the new device and in this example case it also has nosecurity information about the access rights of the Dart applicationattempting to spread its execution to the old device. In this example,the user will be asked via a user interface on each device forpermission for these devices to interoperate according to the accessneeds communicated by the flags in the Dart. FIG. 31 shows, on thebottom right, the state of the devices after permission is granted forthe devices to interoperate at level 7, corresponding to the accessflags set which grants all the access rights specified in the Dartpassed to the old device. After getting permission from the user via theuser interface, both the old and new DartDevices now contain the samelevel 7 list of uids of devices and Darts that are allowed tointeroperate without asking permission. Note that should the NewDartDevice next attempt to send a Dart to any of the other devices onits new level 7 list with the same security access flags requirements,it will be allowed to run without any need to ask the user. In effectthe devices are vouching for all the other devices it knows about. Atrusted team of devices will trust each other, even those that it hasnever directly made contact with before if the team's lists of uids hasbeen distributed across the devices through connections even whereintermittent with other devices of the team. In a preferredimplementation, the lists of uids are synchronized using parts of theSocialSynchronization methodology described elsewhere in this document.

Note that the only administration requiring user involvement orunderstanding to form and maintain the team was the ability to answerquestions directly relating to what kinds of access must be grantedbefore the devices can start interoperating in the needed manner. Thesimple tasks of using the devices and Darts drives the collecting anddistributing of access rights with a small number of easily understoodrequests for permissions from the user.

Revocation of access rights for a device can also be accomplished usingSocialSecurity to spread a list of revoked uids of devices and Darts. Ina preferred implementation, any attempts to interoperate between deviceswhich are one of the devices' revoked lists with the needed rights willnot be permitted and the uids in the lists of teams of the revokeddevices for the appropriate levels of security will be removed.

Additional particular embodiments of the inventive social securityinteroperability model and of the more specific version of that model,the Dart Social Security model, are now set forth. In one embodiment (1), the invention provides a method for automatically and transitivelyspreading the access rights and credentials between interoperabilitydevices, the method comprising: (1) assigning to each interoperabilitydevice a unique id; (2) assigning to each interoperability device aninitial set of access rights; (3) assigning to each interoperabilitysoftware package one or more unique ids and embedding in the package thesets of unique ids and associated access rights needed by all and oreach of the independently executable images that are part of theapplication package; and (4) when two interoperability devices open acommunication channel any existing access rights for device teams orapplications associated with unique ids that are no more restrictivethan those existing for the interoperability of the two devices oneither device are synchronized with those on the other device.

In another embodiment (2) , the invention provides the method of (1) ,wherein one or more of the following are true: (1) the interoperabilitydevice comprises a Dart device; (2) the basis for the unique ids are aninteroperability security procedure; (3) the access rights are asdescribed in interoperability security procedure; (4) the package is aninteroperability software package and/or a Dart; (5) the opening of acommunication channel is initiated as part of a device recruitmentprocedure; and (6) the access rights are synchronized using a SocialSynchronization procedure.

In another embodiment (3) , the invention provides the method of (1) ,wherein the method is carried out in the following steps: (1) initiatinga communication session from an interoperability software packagerunning on an initiating interoperability device to a targetinteroperability device in order to carry out a particularinteroperability purpose; (2) determining if proper access rights existbetween and among the devices for the intended purpose; and (3) if theproper access rights exist, the application software package is allowedto extend its execution to the target device by sending an independentlyexecutable image to the target device and running the image in a contextwith the needed access rights.

In another embodiment (4) , the invention provides the method of (3) ,further comprising: (4) if during execution the independently executableimage attempts an access not allowed by the needed access rights of thecontext in which it is running, the execution will be terminated by theinteroperability engine on the target device.

In another embodiment (5) , the invention provides the method of (4) ,further comprising: (5) if the proper required access rights do notexist, the user or users are presented as necessary on one or bothdevices on a user interface requesting that the proper access rights begranted and or the proper credentials be provided.

In another embodiment (6) , the invention provides the method of (5) ,further comprising: (6) if the proper access rights are granted and/orthe proper credentials are provided, the user may optionally berequested specify if the access rights and credentials are to be storedon the devices for a period of time.

In another embodiment (7) , the invention provides the method of (6) ,wherein the period of time may optionally be selected from the periodsof time consisting of: (i) the lifetime of the current communicationsession, (ii) a specified period of time or target date and time whenthe access rights and or credentials storage and or use are to expire;(iii) for an open period of time without defined expiration, or (iv)forever or forever until revoked.

In another embodiment (8) , the invention provides the method of (6) ,further comprising: (7) independently of whether or not the properaccess rights exist or are granted and/or the proper credentials aregathered for the specific purpose and then stored the followingprocedure is optionally preformed: all the now stored access rights andcredentials that are valid forever for all devices and interoperabilityapplications ever stored which are no more restrictive than those forthe same devices or applications as indicated by the unique ids of thedevices and applications associated with the stored access rights andcredentials are then synchronized across both devices.

In another embodiment (9) , the invention provides the method of (5) ,wherein the determining if proper access rights exist between and amongthe devices for the intended purpose), further comprises: (2a)communicating the access rights needs and credentials the softwarepackage will need to carry out the interoperability purpose by theexecuting software package; (2b) inspecting lists of unique ids andassociated access rights and credentials stored on either one of or bothdevices by the interoperability engines of one or both devices todetermine if the devices already have the needed access rights; and (2c)checking to determine if the access rights and/or credentials alreadyexist on the target device and/or on the initiating device that willallow the access needed for the intended purpose.

In another embodiment (10) , the invention provides the method of (8) ,wherein the effect is that the access rights already gathered fordevices and applications at the level or below that is now common toboth devices will now be allowed without user prompting even in caseswhere the unique ids, access rights, and/or credentials where neverdirectly exchanged between two interoperability devices.

In another embodiment (11) , the invention provides the method of (8) ,wherein the step of synchronizing across both devices access rights andcredentials may be reversed such that access rights and/or credentialscan be subsequently revoked by a device with the proper access rightsand/or credentials and/or physical access required by the securitymechanisms of the interoperability players that contain theinteroperability engines running on the devices.

In another embodiment (12) , the invention provides the method of (1) ,wherein the number of user involvements in granting access rightsbetween all combinations of N devices which frequently communicate to beroughly linearly proportional to the number of devices N, rather thanother methods wherein the number of user involvements grows faster as Nincreases.

In another embodiment (13) , the invention provides the method of (8) ,wherein the number of user involvements in granting access rightsbetween all combinations of N devices which frequently communicate to beroughly linearly proportional to the number of devices N, rather thanother methods wherein the number of user involvements grows faster as Nincreases.

In another embodiment (14), the invention provides the method of (12) ,wherein the growth of interactions is substantially proportional to Nwhile conventional other methodologies provide for interactions that areproportional to the (N Choose 2) gamma function because conventionalmethodologies do not have transitivity.

In another embodiment (15) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for automatically and transitively spreading the access rightsand credentials between interoperability devices, the program moduleincluding instructions for: (1) assigning to each interoperabilitydevice a unique id; (2) assigning to each interoperability device aninitial set of access rights; (3) assigning to each interoperabilitysoftware package one or more unique ids and embedding in the package thesets of unique ids and associated access rights needed by all and oreach of the independently executable images that are part of theapplication package; and (4) when two interoperability devices open acommunication channel, any existing access rights for device teams orapplications associated with unique ids that are no more restrictivethan those existing for the interoperability of the two devices oneither device are synchronized with those on the other device.

In another embodiment (16) , the invention provides an apparatus thatautomatically and transitively spreads access rights and credentialsbetween interoperability devices, the apparatus comprising: (1) meansfor assigning to each interoperability device a unique id; (2) means forassigning to each interoperability device an initial set of accessrights; (3) means for assigning to each interoperability softwarepackage one or more unique ids and embedding in the package the sets ofunique ids and associated access rights needed by all and or each of theindependently executable images that are part of the applicationpackage; and (4) means for synchronizing access rights with those on theother devices when two interoperability devices open a communicationchannel so that any existing access rights for device teams orapplications associated with unique ids that are no more restrictivethan those existing for the interoperability of the two devices oneither device are synchronized with those on the other device.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XX. Interoperability Device/DartDevice

Recall that a DartDevice is a highly interoperable device by virtue ofits running a DartPlayer containing a DartEngine and at least onecommunications protocol for connecting to other DartDevices. Aspects ofthe interoperability device and the more particularized DartDevice areillustrated in FIG. 3 (300) and FIG. 4 (3000).

In one embodiment (1) , the invention provides an interoperabilitydevice comprising: (1) a physical Turing complete instruction set basedprocessor coupled with a physical memory capable of performing generalcomputation and input/output operations; (2) at least one means for twoway communication with other devices; and (3) an interoperability enginerunning on the processor and encapsulated in an interoperability playerembodied in an executable format loadable and executable on the device.

In another embodiment (2) , the invention provides an interoperabilitydevice as in (1) , further comprising the interoperability player.

In another embodiment (3) , the invention provides an interoperabilitydevice as in (1) , wherein one or more of the following are true eitheralone or in any combination: (1) the interoperability device is aDartDevice; (2) the Interoperability Engine comprises a DartEngine; and(3) the interoperability player comprises a DartPlayer.

In another embodiment (4) , the invention provides a method foroperating a device in an interoperability mode with other similar ordissimilar devices, the method comprising: (1) providing a physicalTuring complete instruction set executable within a processor coupledwith a physical memory and in combination capable of performing generalcomputation and input/output operations; (2) sending and receivingtwo-way communications between at least the device and another device;and (3) operating an interoperability engine on the processor andencapsulated in an interoperability player embodied in an executableformat loadable and executable on the device.

In another embodiment (5) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for operating a device in an interoperability mode with othersimilar or dissimilar devices, the program module including instructionsfor: (1) providing a physical Turing complete instruction set executablewithin a processor coupled with a physical memory and in combinationcapable of performing general computation and input/output operations;(2) sending and receiving two-way communications between at least thedevice and another device; and (3) operating an interoperability engineon the processor and encapsulated in an interoperability player embodiedin an executable format loadable and executable on the device.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XXI. Interoperability Platform/DartPlatform

Recall that the DartPlatform may be any set of Dart methodologies whichcan carry out the specification, generation, intelligent teaming ofDartDevices and facilitate the spreading and running of Dartinteroperability applications across one or more DartDevices. Aspects ofthe Interoperability Platform, and the more particular DartPlatform ofthe generic interoperability platform, are illustrated in FIG. 3 as wellas in other portions of the detailed description.

Additional embodiments of the interoperability platform, of which theDart Platform is a specific example, are now described. In oneembodiment (1) , the invention provides a system for specifying,building, distributing, and carrying out the intent of aninteroperability software package of independently executable imagesacross a plurality of possibly heterogeneous devices in a secure,reliable, efficient and robust manner, the system comprising: (1) aninteroperability source for specifying an interoperability computerprogram software package; (2) interoperability tools for buildingprocedural instructions; (3) an interoperability format for packaging atleast the procedural instructions; (4) an interoperability instructionset for representing the procedural instructions generated by theinteroperability tools; (5) an interoperability engine for running theinteroperability software package and providing a commoninteroperability infrastructure on all interoperability devices; and (6)device recruitment means for forming, distributing, and maintaining ateam of interoperability devices.

In another embodiment (2) , the invention provides the system of (1) ,wherein the device recruitment means further includes: means for sendingan inspection procedure operative to find a device having a neededresource or capability to at least one reachable device different fromthe initiating source device over at least one communication link, theinspection procedure including inspection procedure instructions codedin an executable form common to both the initiating source device and tothe device the inspection procedure is intended to reach; means forreceiving on the initiating device the return response from each of thereachable devices directly or indirectly over a communication link;means for analyzing, by a procedure executing on the initiating device,the received returns from all responding reachable devices to determinea utilization plan identifying the combination of capabilities andresources of the initiating source device and the responding reachabledevices to best carry out the intent of the software application; andmeans for distributing, by an application program executing on theinitiating device, at least one of executable code, data, content,and/or Dart to at least one of each of the reachable devices identifiedas having a needed resource or capability according to the identifiedutilization plan.

In another embodiment (3) , the invention provides the system of (1) ,wherein one or more of the following optional components are included orused in the system in any combination: (1) an interoperabilityframework; (2) linear tasking means; (3) vertical layering means; (4)application driven power management means; (5) application driven errorrecovery means; (6) an interoperability runtime; (7) an interoperabilityapplication driven runtime; (8) creationism means; (9) virtual pointers;(10) an interoperability security model means; (11) socialsynchronization means; (12) social security means; and (13)interoperability device enabling means.

In another embodiment (4) , the invention provides the system of (1) ,wherein one or more of the following are true in any combination: (1)the system includes a DartPlatform; (2) the interoperability softwarepackage conforms to the interoperability format or is a Dart conformingto the DartFormat; (3) the independently executable images arerenditions; (4) the Interoperability Source is a DartSource; (5) theInteroperability Tools are DartTools; (6) the Interoperability Format isa DartFormat; (7) the Interoperability Instruction Set is aDartInstructionSet; and (8) the recruitment method is a Dart Recruitmentmethod.

In another embodiment (5) , the invention provides the system of (3) ,wherein one or more of the following are true in any combination: (1)the Interoperability Framework is a DartFramework; (2) the LinearTasking is a Dart Linear Tasking; (3) the Vertical Layering is a DartVertical Layering; (4) the Application driven power management is a DartApplication driven power management; (5) the Application driven errorrecovery is as described elsewhere in the detailed description; (6) theInteroperability Runtime is a DartRuntime; (7) the creationism is DartCreationism; (8) the virtual pointers are Dart Virtual Pointers; (9) theInteroperability Security Model is as described elsewhere in thedetailed description; (10) the Social Synchronization is as describedelsewhere in the detailed description; (11) the Social Security is asdescribed elsewhere in the detailed description; and (12) theInteroperability Device Enabling is as described elsewhere in thedetailed description.

In another embodiment (6) , the invention provides the system of (1) ,wherein the system is simple because of one or more of the followingfeatures: a programmer or creator of the application writes and or testsjust one interoperability application that targets an interoperabilityengine and it will provide the following features alone or in anycombination: (1) run well on all suitable interoperability devicesrunning that engine; (2) automatically allow any user interface inputand output to be performed on nearby or remote Personal Computer orintelligently across many devices with superior interface hardware orsoftware resources eliminating the need to separately write andseparately distribute special Personal Computer synchronizationapplications; (3) perform tasks which can only be performed or arebetter performed with two or more cooperating devices; (4) serve asinternet server software without the need to write any more code byplacing the application on an interoperability device running theinteroperability engine that is connected to the internet; (5)distribute itself from device to device in a limited or unlimitedmanner; (6) the use of virtual pointers eliminates the need to writemore tedious code using file operations; (7) make use of virtualpointers to eliminate the need for considering or writing algorithmsused to improve the speed of access to various storage mediums; (8) makeuse of virtual pointers to eliminate the need for thinking about orpartitioning and maintaining the partitioning of a single address spacewhere different memory allocations must be kept from growing into eachother, or be resized dynamically, or copied to larger partitions so theycould grow in size; (9) make use of the hardware abstraction layer codewith makes all communications mechanisms look the same to theapplication regardless of the protocol being used; (10) make additionalcoding unnecessary to support the automatic bridging of protocols sothat devices can be serially teamed even if different protocols are usedbetween different devices; (11) allow the programmer to use the code inthe Interoperability Framework and its use of the rest of theInteroperability Platform to perform much of the most difficult aspectsof interoperability applications without the need to write the code;(12) provide a program development effort that grows linearly with thenumber of devices to support, N, instead of order developmentmethodologies where the effort grows as the square of N or any othermethodology where the growth rate is greater than being linear with N;(13) provide a program testing effort that grows linearly with thenumber of devices to support, N, instead of order developmentmethodologies where the effort grows as the square of N or any othermethodology where the growth rate is greater than being linear with N;and (14) limit the necessary testing is simplified because adaptation islimited to small number of well defined classifications of devices by arenditioning method.

In another embodiment (7) , the invention provides the system of (6) ,wherein the (11) allow the programmer to use the code in theInteroperability Framework and its use of the rest of theInteroperability Platform to perform much of the most difficult aspectsof interoperability applications without the need to write the code, theaspects including one or more of the following in any combination: (a)recruitment's device discovery; (b) recruitment's device teaming; (c)recruitment's spreading of parts of the application intelligently acrossdevices; (d) application level power management; (e) application levelerror recovery; (f) the mixing and matching of event processing unitswherein the functionality is rearranged just by changing the graph ofevent processing units; (g) the dynamic extension of the runtime toinclude separately generated interoperability applications into theruntime of other applications; and (h) maintaining harmonious operationamong the teams of devices using event processing which is automaticallyserialized and synchronized across teams of devices.

In another embodiment (8) , the invention provides the system of (1) ,wherein the system is simple because of one or more of the followingfeatures: the end user of the application has to consider and orunderstand and or administrate any of the following less often, if ever,than the user would have to were conventional static interoperabilitymethods and or conventional procedural interoperability methods usedalone or in any combination: (1) needing to know about, finding, gettingor loading drivers; (2) choices presented wherein a list of devices thatcannot perform the intended functionality are presented along with thosethat can because the underlying system does not yet know what thelimitations of the devices are when the list is created; (3) whatprotocols are to be used, so that the end user does not have topre-select the protocol or communications technology before initiatinginteroperability or knowing what the end user's choices are; (4) therules for securing devices and or data and or code and or contentbecause of the transitivity of rules as carried out by theInteroperability Security Model and or the Social Security Method; (5)the forming of teams of devices to carry out an intended purpose can beautomated by the application; (6) the application components of code anddata and content and meta data are all packaged together according tothe interoperability format and travel together so that the user doesnot have to deal with the preponderance of compatibility errors thatwould otherwise when interdentally generated and or independentlydistributed components that come into contact are incompatible due toone or more of: (a) versioning incompatibilities; (b) specificationmisunderstandings; (c) errors in implementation; (d) shortcuts inimplementation made by the programmers or manufacturers; and (e)necessary components that are found to be missing or otherwiseunreachable; and (7) explicitly installing separately generatedapplications on devices for synchronizing and or backing up data and orcontent across one or more devices.

In another embodiment (9) , the invention provides the system of (1) ,wherein the system is simple because of one or more of the followingfeatures: the manufacturer of the device is simple as compared withconventional static and or procedural methodologies for one or more ofthe following reasons alone or in any combination: (1) much lessdevelopment effort since only the functions and protocols of the actualdevice need to be considered in porting an interoperability engine, andnot all the permutations of possible protocols and characteristics ofother devices because all the adaptation for different devices iscarried out by the applications not by each and every device; (2) no orless need to coordinate, participate or wait for application standardsto be created before designing and or building and or bringing to marketinteroperability devices; (3) lesser support needs becauseinteroperability of devices and applications are easier to configure anduse by the end user; (4) can expose unique capabilities to other deviceswithout writing or distributing components to other types of devices;(5) devices can work with other manufacturers' devices without any orreduced coordination of efforts and or negotiating of contracts; and (6)can advance or cost reduce hardware without the need to rewrite softwareapplications and or distribute new software applications and or softwareupdates to other devices.

In another embodiment (10) , the invention provides the system of (1) ,wherein the system is simple because of one or more of the followingfeatures: the publishing of the interoperability software applicationsis simple as compared with conventional static and conventionalprocedural methods because of one or more of the following alone or inany combination: (1) there is only one package instead of many packagesand parts needed to address the market of heterogeneous devices andcombinations of devices and protocols so that there is a bigger marketaddressable with one product which results in one or more of thefollowing simplifications: (a) lower development efforts; (b) simplerinventory; (c) simpler distribution; and (d) simpler marketing sincemarket is not as segmented by device types and/or communicationsprotocols and or processor types, and/or screen sizes; (2) simplerpricing and promotion models; (3) more sales with less effort; (4)easier use means simpler support; (5) distribution of digital code andor data and/or content and/or packages thereof can be performed directlyfrom device to device; and (6) applications or teasers about theapplications can be distributed directly from device to device withoutthe need to find other ways to communicate the benefits, existence oractual software directly to the potential users of a service accessiblethrough the applications or directly to potential purchasers ofsoftware.

In another embodiment (11) , the invention provides the system of (1) ,wherein the system is secure for one or more of the following reasons:(a) the engine provides a protective sandbox by way of the checking ofall application memory accesses for violations, and limiting directapplication access to storage and other aspects of the device'sresources so that interoperability applications code cannot be used todamage the device, its data, or content or run native code to get aroundthe sandbox; therefore, only the limited code of the engine needs tosecured against virus or other malicious attacks, and not all of thealmost unlimited application code which can be written to run on thedevices; (b) viruses cannot propagate easily from devices with onenative device processor type to devices with other native processors;since having the same processor is no longer necessary to ensureinteroperability, more types of processors are likely to be deployed,making the world of interoperability devices more secure from the spreadof viruses and other malicious software; (c) the Social Security Modelis so simple to administrate and use that people will actually leavesecurity on, rather than turn it off to avoid the administrationotherwise necessary with conventional security methods; and (d) theInteroperability Security Model is implemented entirely or mostly inportable source code that can be thoroughly implemented and debuggedonce, greatly decreasing the likelihood of errors.

In another embodiment (12) , the invention provides the system of (1) ,wherein the system is reliable and or robust for one or more of thefollowing reasons in any combination: (a) applications running on one ormore devices are often or always communicating with parts of an initialpackage spread via the recruitment and renditioning methodologies so asto enjoy a reliably of interoperability greater than that whereindependently developed and or independently distributed applicationsused to perform the interoperability; (b) the same portable source codeof the Interoperability Engine is used on all devices largelyeliminating the problems of implementation and or misunderstandings ofthe specification or other problems associated with independentlydeveloped and distributed source or code used to enable interoperabilitybetween or across devices; (c) the Interoperability Framework providesmuch of the common interoperability functionality across applicationsand devices so that there is more testing of the Framework code andfewer separate designs and implementations, leading to fewer bugs orinteroperability issues; (d) Linear Tasking and or Vertical Layering andor the serialization and synchronization methods of Recruitment ensuresa largely deterministic order of processing of events driving theapplication and device operations, leading to fewer possiblepermutations of operating order that might otherwise cause errors; (e)application level error recovery is built into the InteroperabilityPlatform, from the Interoperability Source, Interoperability Tools, theInteroperability Runtime and the Interoperability engine to ensure thatintermittent interruptions in communications can often be elegantlyhandled without the need for the application to stop all operations orterminate, or be explicitly reset; and (f) the Interoperability SecuritySystem and or the sandbox and or the access rights enforcementimplemented in the Interoperability Engine can blocks viruses or otherapplications from harming the device or its data whether the potentialharm is intentional or unintentional.

In another embodiment (13) , the invention provides the system of (1) ,wherein the system is efficient for one or more of the following reasonsalone or in any combination: (a) only the code, data and content neededfor a particular target device and or particular task needs to be sentover a communications channel and processed by a target device by thevirtues of the Recruitment and or Renditioning and or Creationismmethods for spreading code, data and content; (b) only one applicationpackage is needed to effect even complex multiple deviceinteroperability; (c) the Part images of an interoperability applicationthat conforms to an interoperability format are often shared betweenseparately executable images of the application so that it is notnecessary to duplicate much of the data shared by the separatelyexecutable images; (d) Virtual Pointers automatically make advantageoustradeoffs between the use of main memory and physical storage to makeapplications able to run with less physical memory than would otherwisebe necessary; (e) Virtual Pointers automatically make advantageoustradeoffs between the use of main memory and physical storage to makeapplications able to run faster than if conventional memory and/orstorage methods were employed; and (f) Virtual Pointers make intelligentadvantageously efficient tradeoffs of memory requirements and speed ofoperation based on the access patterns expected in applications and thephysical characteristics of the main memory and storage of eachparticular device.

In another embodiment (14) , the invention provides the system of (1) ,wherein the software application running on more than one device is atleast partially performing the interoperability operations on two ormore devices with code and/or data and/or content that were originallypart of a single software package on the initiating device so as toenjoy a reliably of interoperability greater than that whereindependently developed and/or independently distributed applicationsare used to perform the interoperability operations.

In another embodiment (15) , the invention provides the system of (3) ,wherein the (1) interoperability framework further comprises a computerprogram or computer program product for executing within a processorlogic and associated memory and including a plurality of computerprogram code instructions implementing a procedure to establish aninteroperability framework.

In another embodiment (16) , the invention provides the system of (3) ,wherein the (2) linear tasking means further comprises a computerprogram or computer program product for executing within a processorlogic and associated memory and including a plurality of computerprogram code instructions implementing a procedure performing lineartasking.

In another embodiment (17) , the invention provides the system of (3) ,wherein the (3) vertical layering means further comprises a computerprogram or computer program product for executing within a processorlogic and associated memory and including a plurality of computerprogram code instructions implementing a procedure performing verticallayering.

In another embodiment (18) , the invention provides the system of (3) ,wherein the (4) application driven power management means furthercomprises a computer program or computer program product for executingwithin a processor logic and associated memory and including a pluralityof computer program code instructions implementing a procedureperforming application driven power management.

In another embodiment (19) , the invention provides the system of (3) ,wherein the (5) application driven error recovery means furthercomprises a computer program or computer program product for executingwithin a processor logic and associated memory and including a pluralityof computer program code instructions implementing a procedureperforming application driven error recovery.

In another embodiment (20) , the invention provides the system of (3) ,wherein the (6) interoperability runtime further comprises a computerprogram or computer program product for executing within a processorlogic and associated memory and including a plurality of computerprogram code instructions implementing a procedure establishing ainteroperability runtime.

In another embodiment (21) , the invention provides the system of (3) ,wherein the (7) interoperability application driven runtime furthercomprises a computer program or computer program product for executingwithin a processor logic and associated memory and including a pluralityof computer program code instructions implementing a procedureestablishing and performing an interoperability runtime.

In another embodiment (22) , the invention provides the system of (3) ,wherein the (8) creationism means further comprises a computer programor computer program product for executing within a processor logic andassociated memory and including a plurality of computer program codeinstructions implementing a procedure performing creationism.

In another embodiment (23) , the invention provides the system of (3) ,wherein the (9) virtual pointers further comprises a computer program orcomputer program product for executing within a processor logic andassociated memory and including a plurality of computer program codeinstructions implementing a procedure performing creationism.

In another embodiment (24) , the invention provides the system of (3) ,wherein the (10) interoperability security model means further comprisesa computer program or computer program product for executing within aprocessor logic and associated memory and including a plurality ofcomputer program code instructions implementing a procedure establishingan interoperability security model.

In another embodiment (25) , the invention provides the system of (3) ,wherein the (11) social synchronization means further comprises acomputer program or computer program product for executing within aprocessor logic and associated memory and including a plurality ofcomputer program code instructions implementing a procedure performingan interoperability security model.

In another embodiment (26) , the invention provides the system of (3) ,wherein the (12) social security means further comprises a computerprogram or computer program product for executing within a processorlogic and associated memory and including a plurality of computerprogram code instructions implementing a procedure establishing socialsecurity.

In another embodiment (27) , the invention provides the system of (3) ,wherein the (13) interoperability device enabling means furthercomprises a computer program or computer program product for executingwithin a processor logic and associated memory and including a pluralityof computer program code instructions implementing a procedureperforming social security.

In another embodiment (28) , the invention provides a method forspecifying, building, distributing, and carrying out the intent of aninteroperability software package of independently executable imagesacross a plurality of possibly heterogeneous devices in a secure,reliable, efficient and robust manner, the method comprising: (1)generating or providing an interoperability source for specifying; (2)generating or providing interoperability tools for building proceduralinstructions; (3) generating or providing an interoperability format forpackaging at least the procedural instructions; (4) generating orproviding an interoperability instruction set for representing theprocedural instructions generated by the interoperability tools; (5)generating or providing an interoperability engine for running theinteroperability software package and providing a commoninteroperability infrastructure on all interoperability devices; and (6)performing device recruitment for forming, distributing, and maintaininga team of interoperability devices.

In another embodiment (29) , the invention provides a method as in (28), further comprising: sending an inspection procedure operative to finda device having a needed resource or capability to at least onereachable device different from the initiating source device over atleast one communication link, the inspection procedure includinginspection procedure instructions coded in an executable form common toboth the initiating source device and to the device the inspectionprocedure is intended to reach; receiving on the initiating device thereturn response from each of the reachable devices directly orindirectly over a communication link; analyzing, by a procedureexecuting on the initiating device, the received returns from allresponding reachable devices to determine a utilization plan identifyingthe combination of capabilities and resources of the initiating sourcedevice and the responding reachable devices to best carry out the intentof the software application; and distributing, by an application programexecuting on the initiating device, at least one of executable code,data, content, and/or Dart to at least one of each of the reachabledevices identified as having a needed resource or capability accordingto the identified utilization plan.

In another embodiment (30) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for specifying, building, distributing, and carrying out theintent of an interoperability software package of independentlyexecutable images across a plurality of possibly heterogeneous devicesin a secure, reliable, efficient and robust manner, the program moduleincluding instructions for: (1) generating or providing aninteroperability source for specifying; (2) generating or providinginteroperability tools for building procedural instructions; (3)generating or providing an interoperability format for packaging atleast the procedural instructions; (4) generating or providing aninteroperability instruction set for representing the proceduralinstructions generated by the interoperability tools; (5) generating orproviding an interoperability engine for running the interoperabilitysoftware package and providing a common interoperability infrastructureon all interoperability devices; and (6) performing device recruitmentfor forming, distributing, and maintaining a team of interoperabilitydevices.

In another embodiment (31) , the invention provides the computer programproduct of (30) , wherein the instructions for performing devicerecruitment further includes instructions for: sending an inspectionprocedure operative to find a device having a needed resource orcapability to at least one reachable device different from theinitiating source device over at least one communication link, theinspection procedure including inspection procedure instructions codedin an executable form common to both the initiating source device and tothe device the inspection procedure is intended to reach; receiving onthe initiating device the return response from each of the reachabledevices directly or indirectly over a communication link; analyzing, bya procedure executing on the initiating device, the received returnsfrom all responding reachable devices to determine a utilization planidentifying the combination of capabilities and resources of theinitiating source device and the responding reachable devices to bestcarry out the intent of the software application; and distributing, byan application program executing on the initiating device, at least oneof executable code, data, content, and/or Dart to at least one of eachof the reachable devices identified as having a needed resource orcapability according to the identified utilization plan.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

XXII. Virtual Pointers

Conventional programs are most often compiled and linked targeting aconventional processor which can directly address just one linear dataaddress space. Often processors implement virtual memory to allowprograms and operating systems to directly address main memory largerthan the physical memory by implementing a system of virtual memorywhere a limited number of real memory blocks, or pages, areautomatically swapped in and out of a larger but slower storage device.This advantageously frees the programmer from having to concern herselfwith logic otherwise needed to directly manage the swapping between thedirectly addressable fast main memory and slower but larger storagedevices.

Still programmers must often concern themselves with memory managementlogic to keep separate data structures from addressing conflicts witheach other as they dynamically change size during execution of programs.Dynamically dividing up and managing the shared use of a single addressspace is often complex and prone to bugs, even where virtual memorytechniques are used to create a larger effective data address space.

Another limitation of conventional virtual memory techniques is that theprocessor only supports real memory pages of a fixed size regardless ofthe characteristics of the program that is running, or in a way thatrequires one size of real memory changes to be used for multipleprograms where the size does optimally match the needs of all thesimultaneously loaded programs or sections of programs.

In one implementation of the invention, more complex addressing logic isused either in a hardware processor or a software instruction setexecution engine to provide multiple independent address spaces for useby each program, and to allow the number of real memory pages and theirsize to advantageously vary depending on the physical size of availablereal main memory, speed and performance characteristics of the mainmemory, and the expected access patterns of specific programs tospecific data sets. Inventive programming language extensions aresupported in the source code, compiler and linker to provide for the useof the inventive VirtualPointers. Using VirtualPointers has thefollowing benefits:

1. Multiple large independent address spaces. Each dynamically resizeddata structure can be kept in its own different VirtualPointer addressspaces so that no conflicts with data structures in other address spacescan occur.

2. Application specific control over the number of real memory pages foreach individual independent address space. The optimal number of realmemory pages can be set according to the expected access patterns tolimit the amount of real memory required, an or to limit the number ofslow block reads and writes that are needed to swap data between realmemory and storage. For example, if a large amount of data is alwaysaccessed starting at zero and continuing in linear order, more than onememory page would be superfluous. The page size could be set to theoptimal access block size of the actual storage to ensure goodperformance speed, or limit the number of accesses to the storage blocksso as not to wear out the device.

3. Device specific control over real memory page sizes to match storagedevice performance characteristics or improve the lifetime of thestorage device. Often devices use flash memory for storage which have aknown limited number of guaranteed block accesses before they startmalfunctioning. Hard disks often store or cache data in blocks of aspecific size.

4. No need for the programmer to predict or administrate the amount ofmemory needed for data structures or lists because the virtual pointerautomatically logically includes values for an address space larger thanthe expected largest such data structure or list used to access it;

5. Automatic saving of data in a form independent of page sizes ornumber of pages. DartSource can be used to set a parameter of eachVirtualPointer variable which indicates whether the saving of Dart usingthe DartInstructionSet SAVE_INSTRUCTION will automatically save theentire data state of the address space. In one preferred implementation,the values of each VirtualPointer that are to be saved are kept as aDartPart with a sparse list of ranges of values without regard to thepage size or number of pages currently in use by the running Dart. Thismakes it possible for the running of the saved Dart on other deviceswhere there are needs for differing page size and or number of pages.

6. Efficient caching of data from a larger and possibly slower datastore with a minimal amount of precious main memory RAM allowingapplications to run as if they have much larger RAM memories than theyphysically have.

7. Simple and efficient base infrastructure for indexed databaseoperations where the data and indexes are kept in different virtualaddress spaces.

In a preferred implementation VirtualPointers are supported by theDartSource, DartTools, and DartEngine to provide the listed benefits.

In the DartSource each virtual pointer is specified using a #pragmaVirtualPointer (parameters) extension of the C or C++ language withparameters which include:

-   -   1. The name of a pointer variable that will hold the starting        address of a virtual address space when the program starts        execution.    -   2. The number of real memory pages suggested for use.    -   3. A binary flag indicating if the values in the address space        are to be automatically saved along with the application.    -   4. The suggested size of the real memory pages.

The DartTools process the DartSource #pragma statements to create a Dartwhich will load the pointer address value which effectively serves asthe starting offset of the separate address space. This starting offsetis really part of a multi-field address that will be interpreted by theDartEngine to identify a specific virtual pointer.

A Dart Virtual Pointer example is shown in FIG. 32. In a preferredimplementation the address space of a Dart is in units of 32-bit words,rather than the more common 8 bit bytes of most conventional processorswhere the maximum directly addressable space is 2ˆ32 8-bit bytes. TheDartPlatform uses the two most significant bits of a 32-bit address as afield indicating the type of address space to be used. Since theaddressable units of the DartInstructionSet carried out by theDartEngine are 32-bit words, there is no loss in the size of memoryaddressable, which is 2ˆ30 32 bit words. The example in FIG. 32 shows anaddress where the address space type field indicates that this is aVirtualPointer address space. The next five-bit field indicates thatVirtualPointer 1 is being used, meaning that in this implementation,there can be up to 2ˆ5=32 different VirtualPointer address spaces. Theremaining 25-bits are the offset or address of the data item inVirtualPointer 1's address space.

The example in FIG. 32 shows a specific instance of how VirtualPointer1's access is being managed by the DartEngine on a specific device for aspecific address, 0x50001. Since the block (page) size is 64K=2ˆ16words, the upper 9 bits of the 25-bit offset indicate which 64K blockcontains the data value being accessed. In the example shown, neither ofthe two real memory pages currently holds that block. Also neither ofthe flash storage block files for VirtualPointer1 hold the data blockneeded. These files are the result of past accesses which required thereal memory pages for block 100 and 101 to be replaced in the past byother pages. The DartEngine automatically saved the data values of theseblocks in the two Flash memory files before replacing the real memorypages. The actual current data value for this access is still in aVirtualPointer1 Part in the running Dart file. This Part was createdwhen the running Dart was last saved using data values then current inits VirtualPointer1's address space. The access in the example will becompeted when the data values for block 101 are read from the thirdrange in the Part into one of the two real memory pages. If the realmemory page to be replaced has data market as changed or dirty, then anew Flash storage file will be written out to save the current values inthe real memory page before it's values are replaced. The access iscomplete when a pointer to the value for 0x50001 now in real memory, isreturned for processing by the accessing Dart instruction.

Additional aspects and embodiments of virtual pointers and their userelative to other aspects of the invention are now set forth. In oneembodiment (1) , the invention provides a method of providing aplurality of large independent data access address spaces accessiblewithin a software application program which efficiently make use ofphysical memory and or physical storage devices, the method comprising:(1) specifying the properties of one or more independent address spaces;(2) processing computer program code source statements into anexecutable image suitable to run on a particular software engine and orhardware processor which supports virtual pointer functionality; and (3)running the executable image on the particular software engine orhardware processor which executes an instruction set that resolvesaccesses to data referenced by the address space by using a fast accesslimited size main memory and slower access but larger size secondarystorage in an efficient manner.

In another embodiment (2) , the invention provides the method in (1) ,wherein the specifying the properties of one or more independent addressspaces is performed using a computer programming language that supportsstatements conforming to a syntax and semantics for doing so.

In another embodiment (3) , the invention provides the method in (1) ,wherein the processing comprises running software tools which processthe source statements into an executable image suitable to run on aparticular software engine and or hardware processor.

In another embodiment (4) , the invention provides the method of (1) ,wherein the efficiently making use of and the in an efficient manner arecarried out at least in part by: (1) for each independent address space,creating, and maintaining a number of real memory pages all of a singlespecific size; (2) when accesses to data logically in the independentaddress space are made, resolving the access and returning a value orpointer to main memory containing the value to the instruction orprogram making the access, using the following procedure: (a)determining if the data is in one of the main memory pages; (b) if thedata is not in one of the main memory pages performing the followingprocedure: (i) determining if the data is in the secondary storage; (ii)if it is not in the secondary storage, perform one or more of thefollowing procedures: (1) return a default value; (2) return with apointer to a pointer to a main memory data element that contains adefault value; and (3) return with an access to unknown data code orother indicator that there is no known value for the data; (c) if thedata is in the secondary storage, perform the following procedure: (i)selecting a real memory page and starting address that will hold theblock of data that contains the data value needed; (ii) if the selectedreal memory page is marked as dirty then writing out or otherwise causethe values in the selected real memory page to be written to secondarystorage; (iii) reading or otherwise loading the contiguous range ofaddressable data into the real memory page that is the size and has thesame new starting address as the real memory page; (iv) mark the data inthe real memory page as being not dirty; and (v) return the value of thedata now in the selected main memory page or a pointer to the data nowin the selected main memory page. (d) if the data is in one of the mainmemory pages perform the following procedure: (i) if the access is knownto be one that indicates that the consumer of the value may change thevalue of the data, mark the main memory page as dirty; and (ii) returnthe value of the data main memory page or a pointer to the data in themain memory page that is found to include the data.

In another embodiment (5) , the invention provides the method of (4) ,wherein if the data is not in a real memory page and the data is not onthe secondary storage, proceed as if the data was in secondary storage,except that instead of loading the selected real memory page with datafrom the secondary storage, fill the selected real memory page with adefault value.

In another embodiment (6) , the invention provides the method of (5) ,wherein the method is only used if any of the accesses to data in theread memory page is known to be one that indicates that the consumer ofthe value may change the value of the data.

In another embodiment (7) , the invention provides the method of (4) ,wherein the default value is optionally selected to be one of thefollowing values: zero (“0”), one (“1”), all bits set to one, all bitsset to zero, negative one, the most significant bit set to 1 and allothers set to zero, a particular value reserved to indicate anuninitialized value or an unknown value.

In another embodiment (8) , the invention provides the method of (1) ,wherein one or more of the following are true in any combination: (a)the software application program is a DartRendition or a Dart; (b)specifying the properties of one or more independent address space isdone using extensions to the language for interoperability source; (c)the software tools are the interoperability tools or are the DartTools;(d) the particular software engine and or hardware processor is theinteroperability engine or the DartEngine; and (e) the instruction setis an interoperability instruction set or is the DartInstructionSet.

In another embodiment (9) , the invention provides the method of (4) ,wherein the values kept in secondary storage are logically overwrittenby the presence of data stored in a superseding file.

In another embodiment (10) , the invention provides the method of (8) ,wherein the values are initially kept inside a saved DartFormat Imageand when a value is changed by the execution of a Dart, instead ofhaving to rewrite or copy and replace the data values in the DartFormatimage the data is logically overwritten by a descriptor file with thesame starting address as the block of data corresponding to a blockwhich might be loaded into real memory pages.

In another embodiment (11) , the invention provides the method of (1) ,wherein the properties are optionally selected to be one or more of thefollowing in any combination: (a) number of real memory pages; (b) abinary flag indicating if the values*in the address space are to beautomatically saved along with the application; (c) the size of the realmemory pages; and (d) the method for determining the above propertieswhen not explicitly provided in the statement.

In another embodiment (12) , the invention provides the method of (11) ,wherein the real memory page size is determined by one or more of thefollowing factors or any combination thereof: (i) amount of availablereal memory; (ii) block size of flash media or other media type; (iii)random access memory (RAM) buffer size of the physical storage; and (iv)speed of reading or writing of data to main memory and or physicalstorage.

In another embodiment (13) , the invention provides the method of (12) ,wherein the main memory page size matches the size of the physical unitof access of a secondary storage device, or is an integer fractionalsize of the physical unit of access of a secondary storage device.

In another embodiment (14) , the invention provides the method of (1) ,wherein the specification is done via a #pragma statement of a Clanguage or C++ language or extended language for processing by theinteroperability tools or by the DartTools.

In another embodiment (15) , the invention provides the method (of 1),wherein the method advantageously provides one or more of the followingproperties in any combination: (a) multiple large independent addressspaces; (b) application specific control over the number of real memorypages for each individual independent address space; (c) device specificcontrol over real memory page sizes to match storage device performancecharacteristics; (d) no need for the programmer to predict oradministrate the amount of memory needed for data structures or listsbecause the virtual pointer automatically logically includes values foran address space larger than the expected largest such data structure orlist used to access it; (e) automatic saving of data in a formindependent of page sizes or number of pages; (f) effective caching ofdata from a larger and possibly slower data store with a minimal amountof precious RAM allowing applications to run as if they have much largerRAM memories than they physically do; and (g) simple and efficient baseinfrastructure for indexed database operations where the data andindexes are kept in different virtual pointer address spaces.

In another embodiment (16) , the invention provides the method of (1) ,wherein access to data values in the address space is through the use ofone or more of in any combination: (a) reserved pointer types; (b)reserved pointer values; (c) the values used as pointer values; and (d)reserved instruction addressing modes.

In another embodiment (17) , the invention provides the method of (1) ,wherein physical memory and storage resources are one or more of in anycombination: (a) flash memory; (b) hard disk; (c) optical storage media;(d) solid state storage media; and (e) any other physical medium whichis slower than the main memory attached to a processor.

In another embodiment (18) , the invention provides the method of (1) ,wherein efficiency is achieved for one or more of in any combination offactors: (a) a programmer does not need to write explicit code to do oneor more of the following: (i) allocate and reallocate memory in waysthat would otherwise be necessary; (ii) perform or specify explicit fileoperations; and (iii) storing and restoring of data in the independentaddress spaces where such storing is done automatically as part ofsaving the application; (b) a programmer does not need to specify orconsider the amount of main memory needed; (c) a programmer does notneed to specify or consider the speed of the application's operations;(d) a programmer does not need to specify or consider the application oruser interface response times as experienced by a human user; (e) aprogrammer does not need to specify or consider the speed of access todata and or of the process for locating and or collating data; (f)improving the lifetime of secondary storage devices by reducing thenumber of block accesses; and (g) the amount of code needed in anapplication based on the reduced need as in item a) of this list.

In another embodiment (19) , the invention provides the method of (8) ,wherein a SAVE_INSTRUCTION and or BUILTIN_INSTRUCTION is used toautomatically save the data.

In another embodiment (20) , the invention provides the method of (8) ,wherein the data values are saved in a Dart Part which contains possiblysparse sets of contiguous ranges of values.

In another embodiment (21) , the invention provides the method of (20) ,wherein the data values are selectively restored or not restored basedon the value of an autoload flag bit in the flags word of the Part Tablerecord for each VirtualPointer independent address space during loadingof the Dart by a DartEngine.

In another embodiment (22) , the invention provides a computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for providing a plurality of large independent data accessaddress spaces accessible within a software application program whichefficiently make use of physical memory and or physical storage devices,the program module including instructions for: (1) specifying theproperties of one or more independent address spaces; (2) processingcomputer program code source statements into an executable imagesuitable to run on a particular software engine and or hardwareprocessor which supports virtual pointer functionality; and (3) runningthe executable image on the particular software engine or hardwareprocessor which executes an instruction set that resolves accesses todata referenced by the address space by using a fast access limited sizemain memory and slower access but larger size secondary storage in anefficient manner.

In another embodiment (23) , the invention provides a virtual pointerdata structure, comprising: a memory storage; a plurality of pointersstored in the memory storage; the pointers permitting specification andaccess to a plurality of large independent data access address spacesaccessible within a software application program which efficiently makeuse of physical memory and/or physical storage devices; the pointersspecifying the properties of one or more independent address spaces; andthe pointers permitting resolution of access to data referenced by theaddress space by using a fast access limited size main memory and sloweraccess but larger size secondary storage in an efficient manner.

The features and/or elements recited in these exemplary embodiments aswell as of exemplary embodiments described elsewhere in this detaileddescription or in the drawings may be combined in many different ways sothat embodiments recited above are not limitations of the invention andadditional or alternative embodiments having any different combinationsor permutations of the features and elements are also embodiments of theinvention.

Having described many different aspects, features, advantages, andembodiments of the invention, attention is now directed to a couple ofillustrative examples of the inventive technology so that its structure,procedures, and operation may be more readily understood. The examplesalso describe additional features of the invention that may nototherwise be described in the sections above.

XXIII. Example Applications and Application Scenarios

A. Neutrino Detector Application Example

With reference to FIG. 21, there is illustrated an example of anembodiment of the invention showing how the DartPlatform, through theuse of the DartInstructionSet's OEM_BUILTIN_INSTRUCTION, can be used tomake the native functionality of a new type of DartDevice, a homeNeutrino Detector 3500, discoverable and useable by a mobile phoneDartDevice 3600 even when the mobile phone initially has no knowledge ofthe existence or characteristics of the Neutrino Detector 3500.

The home Neutrino Detector 3500 is a device which only existstheoretically today, but should one exist, it will be appreciated thatthere would be no existing standards for interoperating with such adevice. Yet the DartPlatform provides methods for enabling the NeutrinoDetector (ND) to be interoperable with other DartDevices, in this case amobile phone 3600, even though no existing standards or knowledge of thefunctionality of such devices was known to the manufacturer of themobile phone DartDevice 3600.

The manufacturer of the ND is Mirabella Electronics which has beenassigned the unique id assigned to the symbol, MIRABELLA_ELECTRONICS_IDso as to differentiate any unique OEM functions created and used byMirabella Electronics from those created and used by all othermanufacturers. Mirabella, desiring to have its device interoperable withDartDevices, ported the DartEngine as part of a DartPlayer running onthe ND. To expose the device's unique capability of sampling the numberof Neutrinos which passes through its detector in a five second period,the native embedded function, CollectSamples, that causes the collectionand returns the number of Neutrinos which passed through the detector iscalled from the HAL's OEM(<oemId>, <subfunction>) method. Mirabellaelectronics then used the DartSource and DartTools to create a controlpanel Dart which contains a rendition, R1, that displays a userinterface for the control panel, and a rendition, R0, which processesDart specific events of a type reserved for signaling that sampling isto take place.

Every DartDevice with user interface capabilities will normally beshipped with a GetControlPanels Dart resident on it to be used to findall the control panels implemented as Darts reside on reachableDartDevices which are willing to share its control panel Darts withother DartDevices. When the GetControlPanels Dart is started on themobile phone 3600, it sends a GetControlPanelList DartProcedure to runon all devices reachable by any of its communications protocols. In thiscase the Mobile Phone finds and establishes a communications sessionusing the Bluetooth protocol common to both devices, then sends theDartProcedure as part of a RunProcedure type event to the NeutrinoDetector which runs the DartProcedure. The DartProcedure gathers a listof names and descriptions and the unique id of the ControlPanel Dartwhen it is executed on the ND through the use of builtin instructionsthat are part of every DartDevice's DartInstructionSet. The name anddescription of the available control panel Dart resident on the ND aresent back to the Mobile Phone and displayed by the GetControlPanels Dartfor selection by the user. After selecting the control panel from thelist on the Mobile Phone, an event is sent requesting that the ControlPanel Dart identified by its unique id start execution on the ND andthen recruit the Mobile Phone over the established communicationssession, which automatically replaces the GetControlPanel Dart on thephone with a running Rendition R1 sent by the recruiting ND ControlPanelDart's running R0 rendition.

Rendition R1 then displays a big green sample button on the MobilePhone's screen. When the user selects the button, R1 generates an eventmarked for synchronization that request that a sample be taken. Theevent is automatically sent by the DartRuntime to the teamed ND's eventqueue which results in the processing of the event by using theOEM_BUILTIN_INSTRUCTION to carry out the sampling by causing theDartEngine to call the halOEM( ) method with the parameters assigned byMirabella Electronics to cause the CollectSamples( ) native devicefunction and return the number of samples back to the caller of halOEM() which is then returned by the engine to the executing R0 rendition. R0then calls to place a DisplayNumberOfSamples type event which is markedfor synchronization on its queue which results in the placing of theDisplayNumberOfSamples type event on the queue of the teamed MobilePhone. The R1 rendition running on the Mobile Phone then processes theevent and displays the number of detected Neurtrinos carried in aparameter of the DisplayNumberOfSamples type event on the screen.

Thus the example of FIG. 21 demonstrates that the DartPlatform can beadvantageously used to expose and extend capabilities and control ofeven very unique devices to DartDevices with no prior knowledge of theexistence, characteristics or capabilities of the unique devices.

Some of the significant points from the above example and descriptions,include the following:

First, a device whose sole purpose could not be considered by astandards committee was able to become a DartDevice 400, requiring aboutthe same amount of development effort as making any other device aDartDevice 400.

Second, this new device is able to easily expose its unique hardwarecapability to other devices that existed prior to the existence of anydevices of this type. Preexisting DartDevices 400 can interoperate withthe new device, controlling its behaviors, which are new and unique andpreviously unknown to any devices.

Third, the development effort required to make the Neutrino Detector3500 a DartDevice 400, is similar to making any device a DartDevice 400,without any additional work for any other devices, even though thosedevices existed prior to the technology being exposed. That is, thedevelopment effort which rises in an N-squared manner when usingconventional interoperability methodologies is just an N order effortwhen implemented using the methodologies embodied in the DartPlatform.Because the DartApplications' renditions are communicating with otherrenditions from the same original Darts, a high degree of reliability isattained as compared to conventional interoperabilities where therewould be a need to separately implement and distribute interoperabilitycomponents which are much more likely to encounter incompatibilities.

B. Slide Show Application Example—Stages of Development Through Usage

Now we describe in still greater detail an exemplary system, method,computer program and components thereof in the context of an examplethat describes development through usage using a slide show Dartapplication as the principle example application. This slide showexample is only for purposes of explanation and it will be appreciatedthat the same technology can be advantageously employed for virtuallyany software application including software applications that interactwith or utilize or control hardware which needs to run on virtually anydevice or set of devices whether on one device at a time, sequentially,on many devices simultaneously or in parallel, or otherwise.

In this example, DartSouce (FIG. 3 100) files (C++) and theDartFramework (FIG. 3 102) of class definitions and source code (FIG. 3101) designed to be used as an optional basis for expressing thealgorithms and code necessary to build interoperability applicationsWith reference to FIG. 3, the exemplary slide show Dart applicationbegins as DartSource files, 100 on FIG. 3. Application Code 101generally conforms to the standard C++ language. The SlideShowDartSource 100 also includes standard C++ source files which express theDartFramework 102. The DartFramework 102 is a set of class definitionsand code designed to be used as an optional basis for expressing thealgorithms and code necessary to build interoperability applications tobe rendered into the DartFormat 300 and which conforms to theDartRuntime 8000 (See FIG. 15.)

Calls from DartSource 100 to Dart Engine functions are made through theBUILTIN_INSTRUCTION (FIG. 20 670) which is part of theDartInstructionSet. Calls from the C++ source code to the enginefunctions are made though the Compiler 201 generated BUILTIN_INSTRUCTION670 (See FIG. 20), which is part of the DartInstructionSet executed bythe device independent DartEngine 600 that is part of a device specificDartPlayer 500 running on a DartDevice 400 (See FIG. 4).

Extensions have advantageously been made to the C++ language so thatmany types of Resources 103, may be referenced from within theApplication Code 101 and DartFramework 102. This is done by speciallynamed built-in functions, specifically named data structures, and#pragma extensions with keywords that are recognized by the DartCompiler 201.

Resources may include any type of data or code or references to data orcode anywhere in a file or dynamically generated and reachable by theDartTools 200 via a file system (see FIG. 22 612, FIG. 27 5100, FIG. 285000, and FIG. 26) or any form of network access, wireless access, orvia any form of communication. For the example slide show application,Resources 103 may for example include background image files in JPEGformat, sound effect files in PCM format, and/or any pictures slides orimages, video, text, symbols, or even complete Dart files (700) to beincluded as default demonstration slides. Pragma extensions to the C++language may also be used to identify to the DartTools 200 whichfunctions are to be automatically made into DartProcedures 4000 (SeeFIG. 14), or Parts 303 (See FIG. 13) rather than included in theMainCode 2103 and MainData 2102 (See FIG. 13) Parts (FIG. 3 303). Apartnumber( ) built-in function extension to the C++ language may beused in the source code to get the Compiler 201 generated partId 32-bitvalue used to reference the part when used as parameters to functions,or buildin instructions which are part of the DartInstructionSet.

The DartFramework 102, (See FIG. 3 and FIG. 11), advantageously includesa hierarchy of class specifications and implementations. One of the keybase classes is the Gizmo 115 class. Gizmos 115 may be used toencapsulate a single functional unit of processing. Gizmo 115 derivedclasses contain Setup( ) and Process( ) methods, and references to aparent Gizmo and a list of child Gizmos. These references may optionallybe null references.

Rendition 114 classes are inherited from the Gizmo class, and aredesigned to serve as the starting point for a part of an applicationthat can be loaded and run independently of other Renditions 114 in theapplication.

A MasterRendition 113 class inherits from the Rendition 114 class. Thisis the only active Rendition in a MasterDart 230 (See FIG. 12). TheMasterDart 230 is a binary DartFormat 300 file or image output from theDartTools 200 (See FIG. 3 and FIG. 12) which preferentially contains asingle MasterRendition instance. The MasterDart 230 may advantageouslyinclude all the code, data, content, procedures, and resourcesreferenced in the source code, and is automatically loaded when it isplayed on the MasterPlayer 203. After loading the MasterDart 230, theMasterPlayer 203 executes the C++ constructor methods which should berun before the start of the main( ) entry point. The Master Rendition's113 Main( ) method serves as the logical starting point for theMasterDart 230.

FIG. 12 shows an embodiment wherein the DartSource 100 is input to theDartTools 200. The Compiler 201 converts the DartSource into theDartInstructionSet one file at a time into Objects 220, and collectionsof these Objects into Libraries. Alternatively, these Libraries can becreated by a librarian tool. The Objects and/or Libraries 220 can alsoserve as Resources 103 of the DartSource 100 for other DartMasters andDart applications. The operations of the Compiler 201, the optionalLibrarian tool, and the Linker 202 are generically similar to those ofconventional C++ compilers, librarians and linkers now in use, exceptfor certain significant differences. Some of these differences includethe following:

First, the target instruction set is from the DartInstructionSet.Second, the output executable is a MasterDart 230 intended for executionon a MasterPlayer 203, rather than targeting the actual end targetdevices; and, the MasterDart and the MasterPlayer represent componentsunique to the invention. Third, much of the class definition and linkagestructures that are generated during the operations of conventionalcompilers, librarians and linkers that is usually thrown away after useis maintained in the MasterDart for use by the MasterPlayer. The classdefinitions and linkage structures that are retained are also differentfrom those that may transiently exist in conventional compilers,librarians, and linkers. Fourth, the DartTools 200, process extensionsto the source language for: (i) specifying or automatically collectingthe run-time resource requirements that will be needed during execution;(ii) the inclusion in the MasterDart of Resources 103 as Parts 303 (FIG.3, FIG. 13); and a partnumber( ) source code function which provides thepart numbers that serve as identifiers of resources assigned by theDartTools 200 for use by the application procedures. Additionallyinventive is that the resulting MasterDarts and Darts output by theDartTools may contain any number of Renditions and the informationneeded to put together possibly shared Parts in the Dart or MasterDartimage to construct these Renditions as needed. Still farther inventiveis that the Darts and MasterDarts output by the DartTools containprocedures, data, code and content needed to carry out Recruitment tointelligently establish ad-hoc teams of devices, and then carry out theintent of the application according to the DartRuntime.

The Linker 202 FIG. 12 combines and packages the Object/Libraries 220into a single binary image conforming to the DartFormat 300 (FIG. 3).This image is a DartMaster 230. The DartMaster 230 is intended to run ona MasterPlayer 203, which contains a DartEngine 500 (See FIG. 22). ThisDartEngine 500 contains some BuiltinInstruction (See FIG. 25 and FIG.20) support for calling DartEngine system functions that make use of thesymbol table Parts 2100 (See FIG. 13 and FIG. 14), and other meta-dataparts generated by the Compiler 201 and Linker 202. These functions canbe used by the code in the MasterDart to aid in the collection, forexample, of Resources 103, initialization of data, and assemblage ofParts 2100 and parts of Parts 2200 into a Dart 700 or set of Darts 700.These functions can also aid in reducing the size of the code, orincreasing the processing efficiency of the code by, Parts 2100, anddata needed in the Dart or Darts to be generated.

A MasterDart conforms to the DartFormat 300 (See FIG. 3), but may belimited to having just one RenditionTable 2104 referenced by its PartIdin the RenditionsTable 2101. PartIds are integer valued identifiers usedas references to Parts FIG. 12 303 inside a Dart 700. In one embodimentthe PartIds are 32-bit, but identifiers with different bit counts, suchas 16-bit, 24-bit, 40-bit, 48-bit, 64-bit or any other number of bitsmay be used. In one embodiment, only one MasterRendition object instanceis allowed in the DartSource. The DartTools 200 use the information inthe source code for the initialization of the MasterRendition 113 objectinstance and data structures it references to initialize the parametersinside the RenditionsTable 2101 record. Some parameters in theRenditionsTable 2101 record that corresponds to the MasterRendition 113instance may be filled in automatically by the DartTools 200. TheseRenditionsTable parameters include the number of events and files, andthe amount of heap memory that should be initially allocated when theRendition 114 object instance is first loaded and prepared for runningby a DartPlayer 500. Before any Dart 700 can run on any DartPlayer 500,the DartEngine 600 of the DartPlayer 500 running on the DartDevice 400should advantageously first choose and load a Rendition.

The DartPlayer 500, which provides a device specific applicationexecutable shell around the DartEngine 600, can specify a RenditionPartId to run, or it can specify zero (“0”), which in one embodiment isan indication that it is an illegal PartId. If zero is specified, theDartEngine will load the Rendition 114 which has the PartId of the firstrecord of the RenditionsTable 2101.

To find the RenditionsTable, the DartEngine 600 should first load theTrailer 305 (See FIG. 3), which in one embodiment should be the lastfour 32-bit words in the Dart 700 file. The Trailer 305 contains theoffset of the Part Table 304 from the beginning of the Dart 700 file.Each record in the Part TableRecord 314 (See FIG. 14), in the Part Table304 contains a PartId, such as a 32-bit PartId, along with thestartOffset and endOffset from the beginning of the Dart 700 file wherethe contiguous image of the Part 303, is found in the Dart 700 file.

Only one RenditionsTable 2101 FIG. 13 may usually be allowed in aDartFormat 300 file, and it has a permanently assigned PartId valuewhich is used by the engine to find the Part TableRecord 314corresponding to the RenditionsTable 2101 part. The offsets in this PartTableRecord 314 are used to find the RenditionsTable 2101, and theRenditionsTable records include information about linear segments withinin the MainData 2102 and MainCode 2103 parts needed by the Renditionthat is referenced by the PartId in that record. The RenditionsTable2101 record also contains the starting code image address to beginexecution for that Rendition 114 instance. The RenditionTable 2104records are preferentially all just one 32-bit word that holds thePartId of a part that belongs to the Rendition 114 instance.

In one embodiment of the invention, several steps are associated withloading a Dart 230, on a DartPlayer 203. The same steps are used to loada MasterDart on a MasterPlayer as these are just specific instances of aDart and DartPlayer respectively. These steps may include the followingsteps, including steps that are optional but advantageously performed:

-   Step 1. The DartPlayer 203 on the Application Development Device 900    (See FIG. 12), calls the Dart Engine's 600 Init ( ) method (FIG. 24    6000, FIG. 23 4002) with a parameter specifying a Rendition PartId    to start with the value zero.-   Step 2. The DartEngine 600 then reads the Trailer from the end of    the Dart 230 file, and extracts the offset of the Part Table 304.-   Step 3. Each record of the Part Table 304 is read until a record is    found with the pre-assigned fixed value for the PartId of all    RenditionsTables. Only one RenditionsTable with this fixed value    PartId is allowed in any DartFormat image.-   Step 4. The DartEngine 600 then extracts the startOffset and    endOffset of the RenditionsTable 2101 and uses that (the startOffset    and endoffset) to read the first RenditionsTable record.-   Step 5. The first RenditionsTable record is read and the initial    runtime parameters such as the number of files and events to    allocate memory for and the amount of space to allocate for the heap    and stack, are extracted. The RenditionTable 2104 PartId is also    extracted.-   Step 6. The RenditionTable PartId is used to find the startOffset    and endOffset of the RenditionTable 2104 from the Part Table 304    entry for this PartId.-   Step 7. Each record in the RenditionTable 2104 is then read. In one    embodiment, each record is just one 32-bit word with the PartId of a    Part that belongs to the Rendition. Each Part TableRecord 314    contains flags and parameters that govern what happens at load time.    For example, there may be a flag if the Part should be processed in    some manner according to its contentType, parameters, parameters    and/or parameter2 values, or according to some other parameter or    condition. The MainData and MainCode parts are examples of parts    which must be processed upon loading in order to place the necessary    code and data in memory before the Rendition starts executing.-   Step 8. When the required MainCode 2103 Part TableRecord 314 is    found, the contentType will hold the fixed value assigned to the    MainCode 2103 contentType and the Auto-Load flag bit in the flags    parameter word should be set, and parameter 0 and parameters will    hold the first and last DartInstructionSet code image address    contained in the MainCode 2103 part. While most executable images    output from widely available C++ tools will result in a code image    that always starts at the same fixed offset, this may not normally    be the case for Darts because only the linear contiguous regions    necessary for the Renditions that are output as a result of running    the MasterDart 230 or saving a set of Renditions of the Dart for    backup or sent to another teamed device. In other words, Dart code    images do not always start at the same fixed offset and only the    linear contiguous regions necessary for the Renditions that are    output from MasterDart 230 or Dart through the use of Creationism or    the SAVE_INSTRUCITON or the save BUILTIN_INSTRUCTION may normally be    present. Advantageously any Dart 700 that forms other Darts,    generally only includes the linear contiguously addressed sections    of the original code or data output by the DartTools 200 that are    needed for the Renditions actually in the generated Dart. For    example, once the static constructor code of the MasterDart 230 has    been run as part of the load process, there is generally no need for    the constructor code to take up space in Darts generated while    running the MasterDart 230. This is because the generated Darts do    not themselves normally contain the MasterRendition 230. The same is    true for the data in the MainData 2102 Part of Dart files generated    by the execution of MasterDarts 230 or other Darts.-   Step 9. Memory for a Context structure or Context object instance is    allocated to keep track of the memory, access rights, and status,    associated with the Dart that is being loaded. A unique contextId    value is generated to be used as a global reference for calls by the    Dart's code or other Darts that may be running on the same    DartEngine 600 instance. The flags of the Context will limit the    access of the Dart code to the memory that is explicitly allocated    as the data, DartHeap, or DartStack of the particular running Dart.    Also attempts to access functions, memory or files that are not a    part of this Dart or its DartEngine environment, or for which access    rights have not been established, as represented by flag values of    the context, will result in one embodiment in negative valued error    return codes from the DartEngine Process call 4003, which will cause    the Dart's execution to end, before illegal access violations can    take place.-   Step 10. The environmental and initial memory needs expressed in the    parameters of the Part TableRecords 314, and RenditionsTable records    are extracted. If there is a need to allocate or reallocate memory    for any of the engine maintained EventInfo structures, FileInfo    structures, or CommSessionInfo structures, or there are other    resources needed for the expected execution of the Dart to be    loaded, then allocations, reallocations, initializations and the    like are performed to prepare the DartContext and DartEngine    environment that is shared by DartContexts, DartProcedureContexts,    and DartIdleContexts running on the same DartEngine instance.-   Step 11. A SubFile, such as SubFile 698 (See FIG. 26), is    preferentially opened and read to process or execute the Parts of    the DartFormat 300 image File as if they were independent files    without the need to copy the data into a separate file.

The DartPlatform 50 advantageously uses a unique file system. The filesystem is processed 612 (See FIG. 26) to create a file abstractionattached to a file identifier (fileld) value that can be used toreference the files in Dart code or when data is to be read, passed,written or shared.

Several BuiltinInstruction 670 (See FIG. 20) operation code values maybe used for Read, Write, Open, Close, Rename, and Position operations. AFileInfo structure associated with each fileld in a one-to-one fashionkeeps track of the type and form of storage, properties and accessrights as well as the current storage locations, position of a currentpointer into the file and the like.

Advantageously, the device specific halBase object which maps theportable parts of the DartEngine 600 implementation with the small setof functions needed to operate on the DartDevice 400, is kept small andsimple because most operations not related to those of actualindividually physically stored files are virtualized by the portablecode in the DartEngine as shown in FIG. 26.

For the processing of files, only read, write, open, close, position,getPosition, rename and delete methods of the HAL are necessary to beimplemented to port the DartEngine 600 to a new DartDevice 400. Theportable portion of the DartEngine 600 implementation builds file accessabstractions for files that are kept in allocated memory 697 (See FIG.26), or sub-files of other files. Once opened, a SubFile can be readjust like any other file, but the source and boundaries of the data areactually a proper subset of another file previously opened.

Advantageously; a SubFile remains operational even if the source filesare closed before the SubFile. The use of a SubFile for reading the codeof a Dart for execution allows the code to be executed in place, makingit unnecessary to load the code into memory from a file. This isespecially advantageous when the Dart is in ROM or the file storage of aDartDevice is implemented in some other quickly readable memory devicerather than a slow hard disk drive or the like.

If necessary or desired to improve the speed of execution, the codeportion of the Dart associated with the Rendition to be run may beloaded into memory now allocated for holding and running of the code forthe DartContext, and a memory file (FIG. 26 697) or its equivalentfunctionality can be used to execute the actual DartCode.

-   Step 12. In the particularly advantageous embodiment and    implementation being described here, the Dart code generated by the    DartTools 200 for the MasterDart 230 or any other Dart 700 which may    still contain initial constructors may start execution at the    initial constructors procedure that should be called before the    main( ) function is called. The code address where execution is to    start is in one of the 32-bit words of the RenditionsTable record    corresponding to the Rendition instance that is to run.-   Step 13. The DartTools 200 automatically put a BuiltinInstruction    659 at the end of the initial constructor with a sub-function value    which causes the DartEngine to set the starting execution address    and this object instance pointer for continued execution of the    MasterDart 230 instructions once the DartEngine.Process method 4003    is called for the first time.

This is the last step (Step 13) in the creation and loading of a Dartaccording to this particular embodiment of the inventive method andcomputer program.

Dynamic Generation Of Darts And Dart Procedures—Creationism

The capacity for continual dynamic generation of Darts and/orDartProcedures by Darts and their descendant Darts is one of the morepowerful properties of the DartPlatform 50 and Dart technology basedsystems and methods. Embodiments of the DartPlatform 50 are designed andarchitected throughout the system to support this continuous efficientgeneration of Darts from earlier Darts. The methodology for this isreferred to as Creationism.

For example the DartSource 100, through the use of C++ extensions, maycontain specifications for Parts which can be mixed, matched, and sharedacross generations of Renditions in Darts generated from the originalMasterDart 230 or directly by the other DartTools.

Embodiments of the DartFramework 102 implicitly provide the mechanismand method for specifying Renditions, and all the code, data and Partsthat are to be included in each Rendition. This allows the DartTools 200to parse through the dependency trees of objects generated by theCompiler 201 and Linker 202 to form an effective database of thesections of code, data and resources needed for dynamic generation ofDarts and DartProcedures 4000. In a similar fashion the MasterPlayer 203can trace through all the data, code, methods and functions that arereachable at runtime for any set of starting points, and therebyadvantageously form the information needed to generate Darts whichcontain only the necessary data, code and resources required.Unreachable data, code and content are automatically excluded fromDartFormat images based on the contained Renditions starting points anddata values.

Another mechanism of the DartPlatform 50 may advantageously be used tolimit the amount of code and data necessary is the automatic saving ofthe entire or selected data state of a running Dart at the time itgenerates other Darts 700 or DartProcedures 4000, and the automaticreloading of the entire or selected data state as part of theDartEngine's Dart loading steps as enumerated above. Because the datavalues and content of running darts are part of the DartFormat imagethemselves, they are easily saved and restored when run, so there isgenerally no need in generated Darts for the initial constructor andsetup code which often makes up much of the kinds of user-centricapplications that the DartPlatform 50 is optimized and intended for. Forexample, when a MasterDart 230 is first loaded, many constructors arecalled before execution is set to begin at the main( ) function. This istrue of most all C++ generated programs as well as of other differentsoftware or programming languages that may be used with the invention.Advantageously, the MasterDart 230 could then save itself with a minimalset of instructions in the main( ) function using the builtin functionswhich utilized the Builtin/Instruction 659 to access the Save( ) methodof the DartEngine. Since a complete image of the current data values issaved by default, and then restored when the saved Dart 700 is loaded,there is no need to run the constructor code in the saved Dart 700. Forthis reason the MasterDart 230 can direct the saving process so that thecontinuous linear range of the code that executes the initialconstructors that have already run, is not included in any of theRenditions in the generated Dart. In this manner the generated Dart canbe significantly smaller than the MasterDart 230 that generated it. Itis also likely to be much smaller than a similar functionalityconventional application executable image which keeps all the staticconstructor and setup code needed to reestablish data state whenever theapplication is started.

Eliminating the need to include the initial constructor code is just oneof many other mechanisms and methods used to optimize the size,complexity, and computational requirements of generated Darts. Theseoptimizations generally result in smaller size and lower complexity andcomputational requirements than would be required using conventionalmechanisms and methods.

Data member data or member method elimination is performed when aMasterDart 230 or any Dart 700 conforming to the DartFormat 300, runningon a MasterPlayer 203 generates a Dart or set of Darts. Through a userinterface of the MasterPlayer 203, or execution of the Dart playing onthe MasterPlayer 203, any proper subset of Rendition objects can beselected to be included in a Dart 700 to be generated.

Only the Parts and sections of the code and data that are required forthe selected Renditions should usually be placed into the generatedDart(s). This is carried out by the DartTools 200 and MasterPlayer 203by parsing the relationships of the objects both from meta-datagenerated by the Compiler 201 and Linker 202 and by tracing through therun-time data members, procedure methods calls and actual pointervalues, to find all the data members, code members, and methods of eachclass and then dynamically eliminate from the object instances andaccessing code all references and space to and of the data members,structure members, and method members, which are not reachable byrunning the selected Renditions from their starting Process( ) memberfunctions. Thus all the class information is effectively optimized basedon the actual runtime setup of the Dart running on a MasterPlayer 203once all the starting places have been identified for the Dart(s) 700 tobe generated. Also advantageously, all the setup code which is notreachable from a Renditions' starting method are not included in thegenerated Darts.

Often a significant portion of user-centric applications, such as thosethe DartPlatform are primarily intended for, consists of or includesetup code and data for setting initial screen positions for userinterface images, initializing data structures, building graphs ofconnected objects, generating tables, or the like. The DartFramework 102includes in most class definitions a Setup( ) method intended to becalled only by the MasterRendition or nested calls from other Setup( )calls. Advantageously, if the MasterRendition is not selected to beincluded in the generated Dart(s) 700, then none of the Setup( ) methodsand other methods and functions and all the data members of classes orstatic data elements that are not otherwise reachable from the selectedRenditions, will not be included in the generated Dart(s) 700.Advantageously, there is nothing special about the Setup( ) call thatmakes this optimization happen, as any code, data, methods, or the like,not reachable from the runtime starting points selected will not beincluded in the generated Dart(s) 700 by the DartTools 200.

GatherResources Procedure

The GatherResources procedure or method is another procedure intended tobe used in the DartFramework 102 to eliminate unnecessary code and datain generated Dart(s) 700. The Setup( ) method or call tree from calls tothis method of the MasterRendition may call a hierarchy ofGatherResouces( ) related methods and functions to put up userinterfaces or dynamically gather data, content or other Resources at thetime the MasterDart 230 is run on a MasterPlayer 203.

It may also me noted, in conjunction with the description provided here,that the DartTools 200 may include in a MasterDart 230 a number of Partsto hold the symbol tables, class definitions, and other meta-data as maybe needed by the MasterPlayer 203 to perform the just describedoptimizations. If these Parts are not included in any of the selectedRenditions, then advantageously they will not be included in thegenerated Dart(s) 700.

Some Selected Other Uses of MasterDarts

It is normal for (but not necessary for) MasterDarts 230 to containDartPlatform 50 code and data of much greater size and complexity thanthe Dart applications 700 to be generated. This extra code is used forperforming the accessing, requesting, initializing, user interfaces,pre-calculating of tables, assembling of Parts, and the like. ThusMasterDart(s) 230, once compiled by the Compiler 201 and linked by theLinker 202, can be used and reused by a programmer or non-programmer togenerate customized Dart(s) 700 using new parameters, and Resources, orother data which might change with time or according to othercircumstances, such as a stock quote, to generate the Dart applicationsin a very wide variety either automatically, or at a user's requestanytime after being linked by the Linker 202, by running the MasterDart230 on the MasterPlayer 203.

Dynamic Reconfiguration, Generation, Assembly, and Optimization of Parts

It should by now be appreciated, that MasterDarts 230 as well as theDarts generated from other Darts retain their ability to dynamicallyreconfigure, generate, assemble and optimize the selection of Parts andparts of Parts for the generation of descendent child Darts adapted fornew functions and target transports and environments.

Loading of Parts According to contextType Parameter

Once a Rendition of a Dart 230 has been loaded using the RenditionsTablewhich references the RenditionTables, which each reference the Partsthat are included for each Rendition, all the Parts that require loadingare loaded according to their contentType parameter. TheDartInstructionSet and extensions implemented by the BuiltinInstructionof the DartInstructionSet make for the efficient reconfiguring andgeneration of highly optimized Darts 700 and DartProcedures 4000 fromany Dart 700 or Dart procedure 4000. In one preferred implementation, aflag bit in the flags parameter of the Part TableEntry record is setonly for those parts with require contentType specific processing duringloading.

The MasterDart may usually contain the complete code image for all thecode that can be reached starting at the MasterRendition object instancein the MainCode part. Similarly, the MasterDart may usually contain thecomplete data image for all the data that can be reached starting at theMasterRendition object instance. Much of this will be eliminated fromgenerated DartFormat images.

DartFramework, DartRuntime, and DartEngine

The DartFramework 102, DartRuntime 8000, and DartEngine 600 will now bemore fully described. The DartFramework assumes for efficiency andeffective use of the functionality built into the DartEngine, a certainDartRuntime operation; though, it should be appreciated that most anyC++ program can be successfully built and run using most any runtimethat can be or has been implemented on any standard set of C++ tools.One of the primary powers of the DartFramework is largely derived fromits tight integration and use of the instructions, mechanisms,procedures, and functions implemented as part of the DartEngine. Forexample, the decompression and building of a bitmap image from acompressed JPEG file or Part may be performed by compiling C++ sourcecode from the JPEG standard examples available widely on the Internet,but such code would be expected to execute much more slowly than thesimple use of the builtin functions which employs the BuiltinInstruction(also referred to as BUILTIN_INSTRUCTION elsewhere in this document) tocall the DartEngine method to perform the JPEG decompression operationusing the native code generated when the DartEngine source code wascompiled for the actual processor of the DartDevice.

Similarly, the DartFramework 102 takes advantage of an ecosystem orenvironment of built-in functionality throughout in the implementationof the DartPlatform 50, including in some embodiments for example, butnot requiring all or limited to, the actual DartInstructionSet, theDartRuntime 8000 system, the communications subsystem, the eventprocessing system, the file system (See File System 612 in FIG. 26), thegraphics and input systems, the power management system, Dart assemblycompression and encryption mechanisms, and the decomposition, access,protection, digital rights management and generation mechanisms.

It should be appreciated that having a well-defined broad set orcollection of user-centric functionality expressed with the efficiencyof code generated for the native processor embodied in any and everydevice, is an advantageous and in some embodiments key to making thefunctions and adaptations necessary for portable applications that areable to extend themselves to other devices practical.

In particular it should be appreciated that while in theory, any Turingcompliant instruction set could be used to generate any other code orcarry out any algorithm, in practice this approach will never be asexecution, memory, power, or cost efficient as a tightly integratedplatform which provides a relatively complete platform, and where thenecessary computational or memory hungry functionality is implemented ina platform specific fashion as is the case with the DartEngine'sprocessing of the DartInstructionSet.

Generally, in order for a device to be considered a Dart device(DartDevice) 400, a Dart player such as DartPlayer 500, should beinstalled on the device and there should be at least one communicationmechanism supported on the enabled device. The DartPlayer 500 is thedevice specific executable unit that provides the environment andoptional user interfaces necessary to startup, initialize, giveprocessor control to and closedown of a DartEngine 600. The DartEngine600 is built as an instance of two interconnected classes, the portableplayerBase class and the device specific halBase class. When theDartEngine 600 is ported to a new device, a device specific DartPlayer500 application must be built to encapsulate the DartEngine's execution,and a device specific halBase derived class must be built to provide abridge from the platform independent parts of the DartEngine to theactual device operating system and/or hardware.

The DartPlayer 500 main loop flow is shown in FIG. 23. First theDartEngine.Init( ) 6000 function is called with parameters that identifythe Dart 700 to be loaded, if any. Then the DartEngine.Process( ) (FIG.23 4002, FIG. 24 6000) function is called in a loop until a negativevalue is returned causing DartPlayer.Uninit( ) to be called and theDartPlayer 500 application to close down. Positive return values causevalue specific DartPlayer processing to be performed before returning tothe main loop (See 4000 FIG. 23).

Such specific DartPlayer processing includes releasing control to otherapplications or processes that may be running on the device, suspendingthe PlayerEngine thread based on power management values, collectinginputs such as mouse moves and keyboard presses and converting them intocalls to add Events 800 to the DartEngine's EventQueue (See 8002 FIGS.15 and 5000 FIG. 16). If the positive return code value is one of thosethat is reserved to mean that the Dart is finished processing, thenDartProcess.Uninit( ) will be called and the Dart processing will bediscontinued. For example, returning the DONE positive return code valueis the normal manner in which a running Dart will signal its owntermination.

The DartEngine.Init( ) processing is shown in FIG. 24, for calls toDartEngine.Init( ) that include references to a Dart 700 to load. Incases where the DartEngine 600 is called when no Contexts are active,and there is no Dart 700 to load, an IdleProcedure internal to theDartengine will be loaded into an IdleContext instead of the Dart 700.The IdleProcedure operating in the IdleContext is made up of a loop ofDartInstructionSet instructions which performs the processing necessaryto keep the DartEngineEvents 4003 FIG. 23 process active. In onepreferred implementation, the IdleProcedure simply executes theinstructions that cause a call to the EngineEventProcessing function8002 FIG. 12 and FIG. 15. In one preferred implementation, a singleinstance of the DartPlayer 500 can in general handle any number ofexecuting Darts 700 in any of several manners. One way is by callingseparate loops within the same running native processor thread bymodifying the main loop 4000, or by creating any number of separateDartEngine 600 instances, or by using separate threads or anycombination of the above. An alternative is for Darts to be explicitlyloaded and run as a subdart or child of a running parent Dart. Here theevent processing passing through the event processing units will bepassed to the event processing units of the subdart once loaded as if itwere part of the compiled and linked parent Dart. In the preferredimplementation these event processing units are called Gizmos and areinstances of the DartFramework class Gizmo.

Gizmos

Darts 700 themselves can also run other Darts 700 in series, or inparallel, or in a highly cooperative fashion where Darts are run as partof the processing hierarchy of other Darts. The DartFramework 102 may bemade up largely of Gizmo 115 derived objects. These Gizmos areadvantageously configured in a strictly defined linear order accordingto the references in the child and parent Gizmo pointers (See 10000 FIG.18). This order is used for collecting Gizmos into interconnected treegraphs which determine their order of execution and their relationshipsin terms of access rights and view of screen real estate, sense of time,and other environmental characteristics. For example a parent can changethe time or rate of apparent time change for a child or children Gizmosand its descendents in order to control their processing and pace. Allthis is made easier to implement in a robust fashion by strict adherenceto the specified order of processing as shown, in the example of FIG.18.

FIG. 18 shows an example hierarchy of Gizmo 115 derived objects for arunning slide show Dart 700 application. It is important to understandthat the slideshow Dart 700 application is using the DartEngine 600 tocarry out the DartInstructionSet instructions that make up the code ofthe slide show Dart. In particular, note that the DartPlayer 500 alongwith the DartEngine 600 have been compiled, linked, loaded and run usingstandard build tools that target the DartDevice's 400 native processorand operating system. It is the DartEngine 600 that executes the slideshow Dart's 700 code made up from instructions from theDartInstructionSet, which was generated from the DartTools 200. The MenuGizmo at the top of the hierarchy shown in FIG. 18 is first to getcontrol when the DartInstructionSet based Menu.Process( ) method isstarted executing on the engine as a result of a DartPlayer.Process( )call which causes the DartEngine 600 to begin processing the Dart's 700instructions. Note that in the preferred implementation, the Gizmo atthe top of the hierarchy is a Rendition object of a class inheritingfrom the base Gizmo class as in FIG. 11.

In the example of FIG. 18, each rounded corner box represents a Gizmoderived object that handles a unit of user interface for the slide showapplication. The Menu handles the user interface for selecting variousfunctions including adding slides, starting the automatic sequencing ofthe slides, selecting slide transition effects, and the like. One of theoptions that may be selected from the Menu, is the view of the slideshow. Views in this example, may for example include a Hyper-linkedIndex view where the user can click on the name or description of aslide, a thumbnail index view where the user can click on a smallthumbnail view of a slide, and a slide show view where the slides takeup most of the screen and on screen arrows, physical buttons or a mouseor pen are used to move forward and backward through the hierarchy ofslides. The Menu only has one child Gizmo derived object, but when theuser selects a view from the Menu, the pointer to the child Gizmoderived object is changed to the Gizmo that supports the selected view.Thus advantageously all that is needed to change how the user views andnavigates the slideshow is to change one pointer in the Menu Gizmo.

Each view in the example has a list of pointers to the three child Gizmo115 derived objects shown in the row below them in FIG. 18. The childrenare all simple still pictures or slides, shown in the order from left toright of the order of the pointers in each of the child pointer lists ofthe three views. The child pointer list of the Gizmo derived objectlabeled “3” points to three children in the order shown from left toright. The object labeled with a “4” represents a Gizmo 115 derivedobject that encapsulates a separately produced running subdart, includedin the slide show as a single slide, but in fact being capable ofcontaining its own hierarchy of slides which can each themselves containany picture, view or Darts 700 and all the functionality thereof. Theboxes labeled “7” and “9” contain Gizmo 115 derived objects that displayreal-time playback video/audio slides. It is noted that for this slide,that processing of events by all Gizmo 115 derived objects and thecontained Darts 700 of Dart playback Gizmos 115, receive processingcontrol in the exact order of the graph created by the pointers tochildren and parents. This order is explicitly set forth in the sequenceof number values shown inside the boxes in FIG. 18.

Since most all processing by Gizmo 115 derived objects is through a callto its Process( ) method with a pointer to an event to process asparameter, any parent can create or change the parameters or type ofevents seen by its children, or can decide whether or not to pass anevent to be processed its children. So in effect any parent can set theenvironment for all its children. For example a Dart container Gizmoobject does not need to know that it has only been given a portion ofthe screen real estate available to its parent. Similarly, when a Gizmo115 derived object accesses its GetTime( ) method, it will automaticallyrequest the time from its parent rather than from a builtin call to theengine if the parent has signaled that it is a time source for itschildren. This ability of parents to control the runtime environment ofits children, and their children is a very powerful property of theDartFramework 200 and DartRuntime 8000. This makes it easy to build forexample, slide shows of slide shows, collection Darts which just collectDarts and allow the user to select which ones to run, the ability tofetch a control panel in the form of a Dart 700 from a printer or otherDart enabled device and run it in a rectangle embedded inside thefetching Dart.

DartEngine Architecture—Passing of Control

The architecture of the DartEngine 600 facilitates the passing ofcontrol from a Dart container Gizmo 115 derived object, to the topmostGizmo 115 in the hierarchy of the contained Dart by containingBUILTIN_INSTRUCTION functions to allow a containing Gizmo 115 derivedobject to create a DartContext for the contained Dart 700, get the Dart700 loaded into that Context, and then calling the contained Dart's 700Process( ) via a DartEngine's BUILTIN_INSTRUCTION call to the containedDart's Process( ) method of the Gizmo 115 topmost in its hierarchy.

Along with the calls to manage and pass processing to contained Darts700, the DartEngine 600 also makes the transition from the containerDart 700 code to the contained Dart's code more efficient by allowing anEventProcessing structure instance containing the pointer to the Event800 to be processed by the contained Dart, to be in memory accessibledirectly through the use of a special pointer (see FIG. 32 Pointer Type11) to memory effectively shared between all DartContexts. This allowsthe efficient passing of Events 800 and other parameters between Darts700 which each have otherwise separate address spaces.

When a call is made from one Dart 700 to another Dart 700 the DartEngineremembers the calling Dart's DartContext and then starts operating outof the called Dart's DartContext until the called Dart's DartContextexecutes a DoneInstruction when the engine stack is at the same place aswhen the original call was made. Thus the DartEngine 600 makes it veryeasy and efficient for Darts 700 to run Darts 700 that run Darts 700 adinfinitem as part of the Gizmo 115 derived hierarchy much as they wouldif they had been generated by the DartTools as one Dart 700. ContainingDarts can present any environment it wants to contained Darts 700 justas it can to a child Gizmo 115 derived object.

It might be supposed that the simple passing of control down the entirehierarchy would waste processor time in calling Gizmo 115 derivedobjects which require no processing, such as a Gizmo 115 derived objectthat represents a still picture which does not need to be redisplayed.This problem can be militated against in a preferred embodiment by asystem for pruning the tree of Gizmo 115 derived object calls. EachEventType 801 value is made up a category flag field where exactly onebit will be set, and a event subtype field for the specific Event 800 tobe processed belonging to that category. In one embodiment, there aretwo sets of flags corresponding to the category flags of an Event Type801. One flag is set if the Gizmo derived object itself needs to handleevents of this type, and the other flag is set if any of its descendentsneeds to see the specific category of Event 801. In one embodiment, inevery pass through the hierarchy, the Gizmo 115 derived objects set theflags for themselves and collect a logically OR'ed collection of theflags set by all the children that it calls. Thus a Gizmo always knowsif its children need to be called or not called depending on thecategory of the Event 800 to be processed. Categories are defined bysets of EventsTypes to make the elimination of unnecessary callsefficient. For example, Events 800 having to do with user input, do notneed to be passed down trees of Gizmo 115 derived object which don'tcontain any Gizmo 115 derived objects that need to process user input.

DartEngine.Process( ) Processing

Examples of the DartEngine.Process( ) processing is shown in theembodiments of FIG. 23, FIG. 15, and FIG. 16. This drives the mainexecution of the Dart's DartInstructionSet instructions 611. A main loopis executed to get the next instruction from the Dart's instructionstream, decode the opcodes and source and destination fields of theinstructions, check that the addressed source and destination parametersare not outside of the data areas of the Dart's DartContext, then jumpto the device processor native code of the DartPlayer 600 method orfunction that carries out the function called for by the opcode field ofthe instruction. If source or destination fields of the instructionspecify parameters outside of those allowed to be accessed, then nodispatch is made based on the opcodes, and control is returned justafter the DartPlayer.Process( ) call with a specific negative returnvalue that reflects the specific access violation. This will cause theDartPlayer 500 to call DartPlayer.Uninit( ) and end the Dart's executionwith an error message or other optional indicator.

FIG. 22 shows the relationship and some of the functionality of theDartPlayer 500, DartEngine 600 and Hardware Abstraction Layer (HAL) 650.The HAL 650 is the part of the player implemented as a class derivedfrom the halBase class used to encapsulate the device specific accessfunctions needed. Advantageously, the number and complexity of thehalBase functions that are required to be implemented is kept to aminimum so that porting to new DartDevice 400 types is quick and easy.Having a minimalist approach to the halBase design also aids in ensuringthat separately implemented DartEngines 600 attain a very high degree ofcompatibility because they share most of the implementation source codefunctionality. Thus it is highly beneficial to move as muchfunctionality as possible to the portable device independent portions ofthe DartPlayer 500. Ease of porting and a very high degree ofcompatibility are key features of embodiments of the DartPlatform 50that are partly realized through the use of portable code whereverpossible, and the use of the same source code on every device whereverpossible. Thus while recompilation and linking will often be necessaryfor each different type of device, most of the functionality will becompiled from the same source code as the other devices that areexpected to be able to work cooperatively with the device.

Power Management Features of the Architecture

Power management is another feature which though optional, runs throughmany parts of the DartPlatfom 50 architecture. When the DartPlayer 500calls the DartEngine.Process 4003 method, execution of the Gizmo derivedobject hierarchy starts by setting a variable that holds a response timeneeded in milliseconds (or any other units) to the highest valuerepresentable. As Events are processed by the ordered tree of Gizmo 115derived objects, each object uses a BUILTIN_INSTRUCTION to tell theDartEngine 600 how long it can wait until it needs to have its Process() method called again.

For example, for an object that is displaying a motion video, this mightbe 1/30^(th) of a second if this is the frame rate of the video. TheDartEngine 600 collects the lowest response time needs reported by theBUIILTIN_INSTRUCTION which implements gathering of the minimum responsetime needs passed in as a parameter. When control returns to theDartPlayer 500 after making a complete pass through the hierarchy of theGizmo tree graph, the DartPlayer 500 or DartEngine 600 can use thisinformation to block its thread or otherwise signal that power is notneeded for at least the time indicated. Normally the DartPlayer 500 willblock its thread with a timeout equal to the minimum response timecollected. The blocking will also terminate if new input arrives thatneeds to be processed by the Dart 700. It may be appreciated that inmany cases the application running on a processor is the only entitythat knows the real response time needs at all times, and Dartapplications may advantageously be designed and generated in a mannerthat makes it easy for the DartPlayer 500 to receive this informationand make use of it for greatly reducing the power and energy consumptionneeds of devices without greatly degrading the performance of dynamicapplications.

Event Processing Queuing Mechanism

The Event processing queuing mechanism, carried out during theEngineEventProcessing (FIG. 15 8002, FIG. 16 5000) processing may alsokeep track of or monitor response time needs for asynchronous processes(FIG. 16 the contents of the dashed box) that are not part of theGizmo.Process(Eventinfo*) calling hierarchy. In this way, the minimumresponse time needs reported by active Darts reflects the combined needsof the mainstream synchronous Dart application processing beingperformed by the Linear Tasking of the Gizmo tree, and the asynchronousprocesses handled by the Event processing portions of the DartEngine600.

Security and Integrity—Execution Checks Performed During InstructionDecoding

Security, integrity and robustness of the DartPlatform 50 may be atleast partially assured through the use of execution checks performedwhile decoding the instructions of the DartInstructionSet. The addressesof data and source fields of the instructions are tested by the engineto make sure that they refer to data within the range of the executingDarts legally accessible data regions according to the DartContext it isrunning in. Similarly, Dart applications and the DartInstructionSetinstructions that the procedures of the applications are made up of, arelimited to accessing only physical storage that belongs to the runningDart. This is to prevent malicious or malfunctioning Darts from havingthe power to access private data or storage resources that should not beaccessible by the running Darts.

The Hardware Abstraction Layer (HAL) file system naming conventionsassure that Dart applications have no way to specify a file outside ofits legal domain (but see optional deliberate exception). Whenspecifying a file for an operation such as open or delete, the Dartapplication cannot specify full file paths but merely a locationId andnumber, or locationId and terminal name string as shown in 5010 FIG. 28.In this way the physical storage accessible is limited in the HAL to theLocations appearing in 5020 FIG. 28. It is up to the HAL to perform theopen, close, read, write, position, rename and get-position operationsbased only on these parameters to specify files. The one intentional butoptional exception to the no access outside the HAL enforced sandbox, isthat a locationId corresponding to a USER_SPECIFIED_LOCATION mayoptionally be supported in the HAL for providing access to files outsideof the Darts Context limits. It is up to the HAL when opening, deletingor renaming files to explicitly prompt the user for a file selection.This allows applications to import and export files, albeit only withthe implicit permission granted by the involvement of the user.

While the present inventive structure, method, apparatus, computerprograms, and computer program products have been described withreference to a few specific embodiments, the description is illustrativeof the invention and is not to be construed as limiting the invention.Various modifications may occur to those skilled in the art withoutdeparting from the true spirit and scope of the invention as defined bythe appended claims. All references cited herein are hereby incorporatedby reference.

1. An interoperability device comprising: (1) a physical Turing completeinstruction set based processor coupled with a physical memory capableof performing general computation and input/output operations; (2) atleast one means for two way communication with other devices; and (3) aninteroperability engine running on the processor and encapsulated in aninteroperability player embodied in an executable format loadable andexecutable on the device.
 2. An interoperability device as in claim 1,further comprising the interoperability player.
 3. An interoperabilitydevice as in claim 1, wherein one or more of the following are trueeither alone or in any combination: (1) the interoperability device is aDartDevice; (2) the Interoperability Engine comprises a DartEngine; and(3) the interoperability player comprises a DartPlayer.
 4. A method foroperating a device in an interoperability mode with other similar ordissimilar devices, the method comprising: (1) providing a physicalTuring complete instruction set executable within a processor coupledwith a physical memory and in combination capable of performing generalcomputation and input/output operations; (2) sending and receivingtwo-way communications between at least the device and another device;and (3) operating an interoperability engine on the processor andencapsulated in an interoperability player embodied in an executableformat loadable and executable on the device.
 5. A computer programproduct for use in conjunction with a computer system or informationappliance, the computer program product comprising a computer readablestorage medium and a computer program mechanism embedded therein, thecomputer program mechanism comprising: a program module that directs thecomputer system or information appliance to function in a specifiedmanner for operating a device in an interoperability mode with othersimilar or dissimilar devices, the program module including instructionsfor: (1) providing a physical Turing complete instruction set executablewithin a processor coupled with a physical memory and in combinationcapable of performing general computation and input/output operations;(2) sending and receiving two-way communications between at least thedevice and another device; and (3) operating an interoperability engineon the processor and encapsulated in an interoperability player embodiedin an executable format loadable and executable on the device.